2024-09-22T03:12:09.587 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-22T03:12:09.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T03:12:09.707 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757 branch: squid description: crimson-rados/thrash_simple/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore thrashers/simple thrashosds-health workloads/radosbench-high-concurrency} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914757' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: seastore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: min_in: 4 workunit: branch: squid env: CRIMSON_COMPAT: '1' sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 1158 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 0/1 suite: crimson-rados suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwS4KtJmnkd0Y7HAZ6MrpH+nNrcMjtptn2wen0MGqPAHFxjqhhIVKJvEGPxI2BSN1OQDjX7VHGhpUA9JCy7bPc= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHHZoZ6An1ukvNxbibSa2sboXGvQkbIV5wOVbVMlt0EXrZsXcZ1IPxKY5WrpjGYzmBlAf1cHutOqlMFyZSeAnpc= tasks: - install: null - ceph: conf: osd: debug monc: 20 flavor: crimson - thrashosds: ceph_objectstore_tool: false chance_down: 0 chance_inject_pause_short: 0 chance_reset_purged_snaps_last: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - full_sequential: - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:56:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-22T03:12:09.708 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-22T03:12:09.708 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-22T03:12:09.709 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-22T03:12:09.710 INFO:teuthology.task.internal:Checking packages... 2024-09-22T03:12:09.731 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-22T03:12:09.731 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-22T03:12:09.731 INFO:teuthology.packaging:ref: None 2024-09-22T03:12:09.731 INFO:teuthology.packaging:tag: None 2024-09-22T03:12:09.731 INFO:teuthology.packaging:branch: squid 2024-09-22T03:12:09.731 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:12:09.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-22T03:12:09.955 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373.g4e45e6b4 2024-09-22T03:12:09.957 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-22T03:12:09.963 INFO:teuthology.task.internal:no buildpackages task found 2024-09-22T03:12:09.963 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-22T03:12:10.018 INFO:teuthology.task.internal:Saving configuration 2024-09-22T03:12:10.035 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-22T03:12:10.108 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-22T03:12:10.134 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 03:07:11.385921', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwS4KtJmnkd0Y7HAZ6MrpH+nNrcMjtptn2wen0MGqPAHFxjqhhIVKJvEGPxI2BSN1OQDjX7VHGhpUA9JCy7bPc='} 2024-09-22T03:12:10.155 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 03:07:11.384528', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHHZoZ6An1ukvNxbibSa2sboXGvQkbIV5wOVbVMlt0EXrZsXcZ1IPxKY5WrpjGYzmBlAf1cHutOqlMFyZSeAnpc='} 2024-09-22T03:12:10.155 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-22T03:12:10.179 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-09-22T03:12:10.179 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-22T03:12:10.180 INFO:teuthology.run_tasks:Running task console_log... 2024-09-22T03:12:10.264 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa7e14ec160>, signals=[15]) 2024-09-22T03:12:10.264 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-22T03:12:10.305 INFO:teuthology.task.internal:Opening connections... 2024-09-22T03:12:10.305 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-22T03:12:10.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:12:10.385 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-09-22T03:12:10.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:12:10.461 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-22T03:12:10.468 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-22T03:12:10.486 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-22T03:12:10.487 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-22T03:12:10.544 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-09-22T03:12:10.544 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-09-22T03:12:10.544 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-09-22T03:12:10.544 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-09-22T03:12:10.544 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-09-22T03:12:10.544 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-22T03:12:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-22T03:12:10.546 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-22T03:12:10.571 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-09-22T03:12:10.590 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-09-22T03:12:10.590 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-09-22T03:12:10.648 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-09-22T03:12:10.648 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-09-22T03:12:10.648 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-09-22T03:12:10.648 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-09-22T03:12:10.648 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-09-22T03:12:10.648 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-22T03:12:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-22T03:12:10.649 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-09-22T03:12:10.671 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-22T03:12:10.767 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-22T03:12:10.816 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-22T03:12:10.816 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-22T03:12:10.819 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-09-22T03:12:10.836 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-22T03:12:10.885 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-22T03:12:10.885 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-22T03:12:10.887 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-09-22T03:12:10.925 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-22T03:12:11.027 INFO:teuthology.run_tasks:Running task kernel... 2024-09-22T03:12:11.041 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-22T03:12:11.042 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-22T03:12:11.042 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T03:12:11.042 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T03:12:11.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:12:11.059 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-22T03:12:11.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:12:11.060 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-22T03:12:11.116 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-503.el9.x86_64 2024-09-22T03:12:11.117 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-503.el9.x86_64 2024-09-22T03:12:11.117 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-09-22T03:12:11.118 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-503.el9.x86_64 2024-09-22T03:12:11.119 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-503.el9.x86_64 2024-09-22T03:12:11.119 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-09-22T03:12:13.273 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.3 MB 00:00 2024-09-22T03:12:13.680 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.3 MB 00:00 2024-09-22T03:12:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 38 MB/s | 20 MB 00:00 2024-09-22T03:12:18.005 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-22T03:12:20.383 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-22T03:12:21.817 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 19 kB 00:00 2024-09-22T03:12:22.531 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-22T03:12:22.780 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-22T03:12:24.873 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-22T03:12:26.194 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-22T03:12:28.160 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-22T03:12:30.017 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-22T03:12:30.017 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-22T03:12:30.120 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:12:30.129 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:12:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-22T03:12:30.130 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:12:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 79 M 2024-09-22T03:12:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 126 M 2024-09-22T03:12:30.130 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:12:30.370 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 307 kB/s | 36 kB 00:00 2024-09-22T03:12:31.195 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-22T03:12:31.566 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-22T03:12:31.779 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-22T03:12:32.079 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 19 MB/s | 35 MB 00:01 2024-09-22T03:12:32.079 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:12:32.080 INFO:teuthology.orchestra.run.smithi129.stdout:Total 40 MB/s | 79 MB 00:01 2024-09-22T03:12:32.764 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:12:33.105 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:12:33.105 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:12:33.481 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-22T03:12:33.481 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-22T03:12:33.572 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:12:33.576 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:12:33.576 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-22T03:12:33.576 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:12:33.576 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:12:33.576 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-22T03:12:33.576 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:12:33.576 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-22T03:12:33.577 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-22T03:12:33.577 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-22T03:12:33.577 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:12:33.577 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:12:33.577 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:12:33.577 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-09-22T03:12:33.577 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:12:33.578 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 79 M 2024-09-22T03:12:33.578 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 126 M 2024-09-22T03:12:33.578 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:12:33.818 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 322 kB/s | 36 kB 00:00 2024-09-22T03:12:34.067 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:12:34.067 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:12:34.210 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-22T03:12:34.568 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-22T03:12:34.885 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-22T03:12:34.886 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:12:34.886 INFO:teuthology.orchestra.run.smithi144.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-22T03:12:35.573 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:12:35.939 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:12:35.940 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:12:36.487 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:12:36.912 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:12:36.912 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:12:37.275 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:12:37.387 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:12:39.364 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:12:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:12:40.104 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:12:40.136 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:12:40.255 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:12:42.846 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:12:42.981 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:12:45.071 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:12:45.205 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:12:47.831 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:12:47.887 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:12:50.058 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:12:52.713 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:28.119 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:28.159 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:13:29.364 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:13:29.722 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:13:29.798 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:13:29.799 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:13:29.799 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-22T03:13:29.799 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T03:13:29.799 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T03:13:29.799 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-22T03:13:29.799 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T03:13:29.799 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:13:29.858 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:13:29.859 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:13:29.859 DEBUG:teuthology.orchestra.run.smithi129:> sudo rpm -qi grub2-tools 2024-09-22T03:13:29.899 INFO:teuthology.orchestra.run.smithi129.stdout:Name : grub2-tools 2024-09-22T03:13:29.899 INFO:teuthology.orchestra.run.smithi129.stdout:Epoch : 1 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Version : 2.06 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Release : 82.el9 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: x86_64 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Group : Unspecified 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Size : 8274670 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:License : GPLv3+ 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Packager : builder@centos.org 2024-09-22T03:13:29.900 INFO:teuthology.orchestra.run.smithi129.stdout:Vendor : CentOS 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout:Summary : Support tools for GRUB. 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout:Description : 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout:hardware devices. 2024-09-22T03:13:29.901 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:13:29.902 INFO:teuthology.orchestra.run.smithi129.stdout:This subpackage provides tools for support of all platforms. 2024-09-22T03:13:29.902 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-22T03:13:29.903 INFO:teuthology.task.kernel:Updating grub on smithi129 to boot 5.14.0-508.el9.x86_64 2024-09-22T03:13:29.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-22T03:13:30.599 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-09-22T03:13:31.188 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:31.625 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:31.625 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:13:31.625 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:13:31.625 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:13:31.932 INFO:teuthology.orchestra.run.smithi129.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-22T03:13:31.966 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-09-22T03:13:31.968 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-22T03:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.SqD4J3Pn0k 2024-09-22T03:13:31.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SqD4J3Pn0k 2024-09-22T03:13:32.051 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.SqD4J3Pn0k 2024-09-22T03:13:32.189 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.SqD4J3Pn0k is 6KB 2024-09-22T03:13:32.237 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.SqD4J3Pn0k 2024-09-22T03:13:32.252 DEBUG:teuthology.orchestra.run.smithi129:> sudo /bin/ls /boot/loader/entries || true 2024-09-22T03:13:32.324 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-22T03:13:32.324 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-22T03:13:32.324 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-22T03:13:32.324 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-22T03:13:32.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-22T03:13:32.472 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:13:32.472 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:13:32.472 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:13:32.472 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-22T03:13:32.472 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-22T03:13:32.473 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-22T03:13:32.473 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-22T03:13:32.473 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:13:32.473 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:13:32.483 DEBUG:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2024-09-22T03:13:32.869 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:13:32.952 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:13:32.952 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:13:32.952 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-22T03:13:32.952 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T03:13:32.952 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T03:13:32.952 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-22T03:13:32.953 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T03:13:32.953 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:13:33.010 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:13:33.010 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:13:33.011 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-09-22T03:13:33.043 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 82.el9 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8274670 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-22T03:13:33.044 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:13:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-09-22T03:13:33.046 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-22T03:13:33.046 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-508.el9.x86_64 2024-09-22T03:13:33.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-22T03:13:33.776 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-09-22T03:13:35.025 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-22T03:13:35.062 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-09-22T03:13:35.065 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-09-22T03:13:35.080 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.brOP32A46I 2024-09-22T03:13:35.080 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.brOP32A46I 2024-09-22T03:13:35.147 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.brOP32A46I 2024-09-22T03:13:35.282 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.brOP32A46I is 6KB 2024-09-22T03:13:35.331 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.brOP32A46I 2024-09-22T03:13:35.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-09-22T03:13:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-22T03:13:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-22T03:13:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-22T03:13:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-22T03:13:35.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-22T03:13:35.573 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-09-22T03:14:02.507 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-22T03:14:02.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-22T03:14:02.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:14:05.578 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-22T03:14:05.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-22T03:14:05.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:14:21.039 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-22T03:14:24.114 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-09-22T03:14:30.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-22T03:14:30.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:14:33.103 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-22T03:14:33.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-22T03:14:33.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:14:45.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-22T03:14:45.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:15:03.535 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-22T03:15:18.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-22T03:15:18.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:15:18.966 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-22T03:15:19.351 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-22T03:15:19.351 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-22T03:15:19.352 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-22T03:15:19.366 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-508.el9.x86_64 2024-09-22T03:15:19.366 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-22T03:15:19.366 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T03:15:19.366 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-22T03:15:20.368 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-22T03:15:20.368 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T03:15:20.446 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-09-22T03:15:20.478 DEBUG:teuthology.parallel:result is None 2024-09-22T03:15:33.118 DEBUG:teuthology.orchestra.remote:timed out 2024-09-22T03:15:45.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-22T03:15:45.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:15:45.592 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-22T03:15:45.994 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-22T03:15:45.994 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-508.el9.x86_64"... 2024-09-22T03:15:45.995 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-22T03:15:46.011 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-508.el9.x86_64 2024-09-22T03:15:46.011 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-22T03:15:46.011 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T03:15:46.011 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-22T03:15:47.012 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-22T03:15:47.012 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T03:15:47.345 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-09-22T03:15:47.372 DEBUG:teuthology.parallel:result is None 2024-09-22T03:15:47.373 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-22T03:15:47.380 INFO:teuthology.task.internal:Creating test directory... 2024-09-22T03:15:47.381 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T03:15:47.384 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T03:15:47.401 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-22T03:15:47.457 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-22T03:15:47.532 INFO:teuthology.task.internal:Creating archive directory... 2024-09-22T03:15:47.532 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T03:15:47.534 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T03:15:47.574 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-22T03:15:47.760 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-22T03:15:47.760 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T03:15:47.763 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T03:15:47.797 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:15:47.801 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:15:47.808 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:15:47.813 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:15:47.814 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-22T03:15:47.824 INFO:teuthology.task.internal:Configuring sudo... 2024-09-22T03:15:47.824 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T03:15:47.853 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T03:15:47.889 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-22T03:15:47.902 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-22T03:15:47.903 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T03:15:47.921 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T03:15:47.946 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:15:48.015 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:15:48.069 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:15:48.148 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:15:48.193 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:15:48.193 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T03:15:48.257 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:15:48.301 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:15:48.354 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:15:48.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:15:48.480 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:15:48.480 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T03:15:48.547 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-22T03:15:48.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-09-22T03:15:48.594 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:15:48.624 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:15:48.938 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-22T03:15:48.955 INFO:teuthology.task.internal:Starting timer... 2024-09-22T03:15:48.955 INFO:teuthology.run_tasks:Running task pcp... 2024-09-22T03:15:49.001 INFO:teuthology.run_tasks:Running task selinux... 2024-09-22T03:15:49.066 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-22T03:15:49.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-09-22T03:15:49.152 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-09-22T03:15:49.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-09-22T03:15:49.221 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-09-22T03:15:49.223 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-22T03:15:49.223 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-09-22T03:15:49.245 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-09-22T03:15:49.246 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-09-22T03:15:49.265 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-09-22T03:15:49.265 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-09-22T03:15:49.265 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:15:49.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:15:49.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:15:49.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:15:49.339 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-22T03:15:49.339 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-09-22T03:15:49.382 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-09-22T03:15:49.422 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-22T03:15:49.432 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-22T03:15:49.443 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-22T03:15:49.443 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi129.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-22T03:19:30.305 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-09-22T03:19:30.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-22T03:19:30.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:19:30.382 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-22T03:19:30.463 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-22T03:19:30.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-22T03:19:30.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:19:30.538 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-22T03:19:30.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-22T03:19:30.622 INFO:teuthology.run_tasks:Running task clock... 2024-09-22T03:19:30.632 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-22T03:19:30.632 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T03:19:30.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:19:30.634 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T03:19:30.635 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:19:30.663 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-22T03:19:30.678 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-22T03:19:30.692 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-22T03:19:30.702 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-09-22T03:19:30.707 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-22T03:19:30.714 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-09-22T03:19:30.727 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-22T03:19:30.732 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-09-22T03:19:30.741 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-22T03:19:30.742 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-09-22T03:19:30.756 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-22T03:19:30.770 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-22T03:19:30.801 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:19:30.805 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:19:30.805 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-22T03:19:30.805 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.805 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.805 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.805 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.824 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:19:30.827 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:19:30.827 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-09-22T03:19:30.827 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.828 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.828 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.828 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:19:30.828 INFO:teuthology.run_tasks:Running task install... 2024-09-22T03:19:30.846 DEBUG:teuthology.task.install:project ceph 2024-09-22T03:19:30.846 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T03:19:30.847 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-22T03:19:30.847 INFO:teuthology.task.install:Using flavor: crimson 2024-09-22T03:19:30.853 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-22T03:19:30.853 INFO:teuthology.task.install:extra packages: [] 2024-09-22T03:19:30.853 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'tag': None, 'wait_for_package': False} 2024-09-22T03:19:30.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:19:30.856 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'tag': None, 'wait_for_package': False} 2024-09-22T03:19:30.856 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:19:31.006 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/ 2024-09-22T03:19:31.006 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-22T03:19:31.014 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/ 2024-09-22T03:19:31.014 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-22T03:19:31.190 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-22T03:19:31.191 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:19:31.191 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-22T03:19:31.193 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-22T03:19:31.193 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:19:31.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-22T03:19:31.219 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-22T03:19:31.219 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e45e6b490e6cbe082cdf155fea780438ec53fc7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-22T03:19:31.220 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-22T03:19:31.221 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e45e6b490e6cbe082cdf155fea780438ec53fc7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-22T03:19:31.287 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-22T03:19:31.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-22T03:19:31.362 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-22T03:19:31.364 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-22T03:19:31.427 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-09-22T03:19:31.428 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-09-22T03:19:31.429 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-09-22T03:19:31.430 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-09-22T03:19:31.658 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-09-22T03:19:31.664 INFO:teuthology.orchestra.run.smithi144.stdout:45 files removed 2024-09-22T03:19:31.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw 2024-09-22T03:19:31.691 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw 2024-09-22T03:19:32.242 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 315 kB/s | 87 kB 00:00 2024-09-22T03:19:32.258 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 308 kB/s | 87 kB 00:00 2024-09-22T03:19:32.526 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-22T03:19:32.551 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-22T03:19:32.801 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 8.3 kB/s | 2.0 kB 00:00 2024-09-22T03:19:32.826 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 8.3 kB/s | 2.0 kB 00:00 2024-09-22T03:19:33.393 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-22T03:19:33.510 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-22T03:19:35.219 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-22T03:19:35.478 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-22T03:19:41.612 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-22T03:19:43.479 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-22T03:19:44.670 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 6.5 MB 00:05 2024-09-22T03:19:45.141 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-22T03:19:46.129 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 19 kB 00:00 2024-09-22T03:19:47.233 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-22T03:19:50.631 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-22T03:19:52.805 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-22T03:19:52.840 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:19:52.843 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:19:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:19:52.843 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:19:52.843 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:19:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 9.9 M 2024-09-22T03:19:52.843 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 19 M 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-22T03:19:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 760 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 501 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.1 M 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-22T03:19:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 161 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 301 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:19:52.846 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:19:52.847 INFO:teuthology.orchestra.run.smithi129.stdout:Install 30 Packages 2024-09-22T03:19:52.847 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 2 Packages 2024-09-22T03:19:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:19:52.847 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 58 M 2024-09-22T03:19:52.847 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:19:53.440 INFO:teuthology.orchestra.run.smithi129.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 12 MB/s | 5.4 MB 00:00 2024-09-22T03:19:53.507 INFO:teuthology.orchestra.run.smithi129.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 377 kB/s | 25 kB 00:00 2024-09-22T03:19:53.616 INFO:teuthology.orchestra.run.smithi129.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 6.9 MB/s | 760 kB 00:00 2024-09-22T03:19:53.707 INFO:teuthology.orchestra.run.smithi129.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 5.4 MB/s | 501 kB 00:00 2024-09-22T03:19:53.816 INFO:teuthology.orchestra.run.smithi129.stdout:(5/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 12 MB/s | 9.9 MB 00:00 2024-09-22T03:19:53.858 INFO:teuthology.orchestra.run.smithi129.stdout:(6/32): python3-ceph-argparse-19.1.1-373.g4e45e 1.1 MB/s | 45 kB 00:00 2024-09-22T03:19:53.959 INFO:teuthology.orchestra.run.smithi129.stdout:(7/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_64 20 MB/s | 5.1 MB 00:00 2024-09-22T03:19:53.983 INFO:teuthology.orchestra.run.smithi129.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 1.1 MB/s | 136 kB 00:00 2024-09-22T03:19:54.009 INFO:teuthology.orchestra.run.smithi129.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 3.2 MB/s | 161 kB 00:00 2024-09-22T03:19:54.034 INFO:teuthology.orchestra.run.smithi129.stdout:(10/32): python3-rados-19.1.1-373.g4e45e6b4.el9 6.2 MB/s | 321 kB 00:00 2024-09-22T03:19:54.061 INFO:teuthology.orchestra.run.smithi129.stdout:(11/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 5.6 MB/s | 301 kB 00:00 2024-09-22T03:19:54.087 INFO:teuthology.orchestra.run.smithi129.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-22T03:19:54.271 INFO:teuthology.orchestra.run.smithi129.stdout:(13/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x 15 MB/s | 19 MB 00:01 2024-09-22T03:19:54.412 INFO:teuthology.orchestra.run.smithi129.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 305 kB/s | 107 kB 00:00 2024-09-22T03:19:54.429 INFO:teuthology.orchestra.run.smithi129.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 285 kB/s | 45 kB 00:00 2024-09-22T03:19:54.455 INFO:teuthology.orchestra.run.smithi129.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-09-22T03:19:54.497 INFO:teuthology.orchestra.run.smithi129.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-22T03:19:54.572 INFO:teuthology.orchestra.run.smithi129.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-22T03:19:54.597 INFO:teuthology.orchestra.run.smithi129.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-22T03:19:54.623 INFO:teuthology.orchestra.run.smithi129.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-22T03:19:54.648 INFO:teuthology.orchestra.run.smithi129.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 476 kB/s | 36 kB 00:00 2024-09-22T03:19:54.673 INFO:teuthology.orchestra.run.smithi129.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-22T03:19:54.732 INFO:teuthology.orchestra.run.smithi129.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-22T03:19:54.757 INFO:teuthology.orchestra.run.smithi129.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 371 kB/s | 26 kB 00:00 2024-09-22T03:19:54.782 INFO:teuthology.orchestra.run.smithi129.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-22T03:19:54.807 INFO:teuthology.orchestra.run.smithi129.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-22T03:19:54.841 INFO:teuthology.orchestra.run.smithi129.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-22T03:19:54.866 INFO:teuthology.orchestra.run.smithi129.stdout:(28/32): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-22T03:19:54.984 INFO:teuthology.orchestra.run.smithi129.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-22T03:19:55.067 INFO:teuthology.orchestra.run.smithi129.stdout:(30/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 16 MB/s | 3.2 MB 00:00 2024-09-22T03:19:55.083 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 9.9 M 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-22T03:19:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 19 M 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 760 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 501 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.1 M 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-22T03:19:55.100 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 161 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 301 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-22T03:19:55.101 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:19:55.102 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:19:55.102 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:19:55.102 INFO:teuthology.orchestra.run.smithi144.stdout:Install 30 Packages 2024-09-22T03:19:55.102 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 2 Packages 2024-09-22T03:19:55.102 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:19:55.102 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 58 M 2024-09-22T03:19:55.102 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:19:55.192 INFO:teuthology.orchestra.run.smithi129.stdout:(31/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 15 MB/s | 3.1 MB 00:00 2024-09-22T03:19:55.284 INFO:teuthology.orchestra.run.smithi129.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-22T03:19:55.289 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:19:55.289 INFO:teuthology.orchestra.run.smithi129.stdout:Total 24 MB/s | 58 MB 00:02 2024-09-22T03:19:55.390 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:19:55.426 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:19:55.426 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:19:56.065 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:19:56.066 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:19:57.323 INFO:teuthology.orchestra.run.smithi144.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 2.7 MB/s | 5.4 MB 00:02 2024-09-22T03:19:57.431 INFO:teuthology.orchestra.run.smithi144.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 233 kB/s | 25 kB 00:00 2024-09-22T03:19:57.523 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:19:57.715 INFO:teuthology.orchestra.run.smithi144.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 2.6 MB/s | 760 kB 00:00 2024-09-22T03:19:57.882 INFO:teuthology.orchestra.run.smithi144.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 2.9 MB/s | 501 kB 00:00 2024-09-22T03:19:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-22T03:19:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:19:58.096 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:19:58.186 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:19:58.213 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:19:58.289 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:19:58.317 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-09-22T03:19:58.352 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-09-22T03:19:58.374 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:19:58.397 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:19:58.580 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:19:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:19:58.642 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-09-22T03:19:58.685 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 10/34 2024-09-22T03:19:58.714 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:19:58.781 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 12/34 2024-09-22T03:19:58.804 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:19:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:(5/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_64 5.3 MB/s | 5.1 MB 00:00 2024-09-22T03:19:58.892 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:19:58.928 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-09-22T03:19:59.007 INFO:teuthology.orchestra.run.smithi144.stdout:(6/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 2.7 MB/s | 9.9 MB 00:03 2024-09-22T03:19:59.033 INFO:teuthology.orchestra.run.smithi144.stdout:(7/32): python3-ceph-argparse-19.1.1-373.g4e45e 235 kB/s | 45 kB 00:00 2024-09-22T03:19:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-22T03:19:59.066 INFO:teuthology.orchestra.run.smithi144.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 2.3 MB/s | 136 kB 00:00 2024-09-22T03:19:59.071 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-22T03:19:59.091 INFO:teuthology.orchestra.run.smithi144.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 2.7 MB/s | 161 kB 00:00 2024-09-22T03:19:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-22T03:19:59.150 INFO:teuthology.orchestra.run.smithi144.stdout:(10/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-22T03:19:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-22T03:19:59.175 INFO:teuthology.orchestra.run.smithi144.stdout:(11/32): python3-rados-19.1.1-373.g4e45e6b4.el9 2.9 MB/s | 321 kB 00:00 2024-09-22T03:19:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-22T03:19:59.209 INFO:teuthology.orchestra.run.smithi144.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-22T03:19:59.215 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-22T03:19:59.309 INFO:teuthology.orchestra.run.smithi144.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 800 kB/s | 107 kB 00:00 2024-09-22T03:19:59.335 INFO:teuthology.orchestra.run.smithi144.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-22T03:19:59.360 INFO:teuthology.orchestra.run.smithi144.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-22T03:19:59.419 INFO:teuthology.orchestra.run.smithi144.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-22T03:19:59.453 INFO:teuthology.orchestra.run.smithi144.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-22T03:19:59.478 INFO:teuthology.orchestra.run.smithi144.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 722 kB/s | 42 kB 00:00 2024-09-22T03:19:59.511 INFO:teuthology.orchestra.run.smithi144.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-22T03:19:59.537 INFO:teuthology.orchestra.run.smithi144.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 611 kB/s | 36 kB 00:00 2024-09-22T03:19:59.558 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-22T03:19:59.562 INFO:teuthology.orchestra.run.smithi144.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-22T03:19:59.602 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:19:59.693 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:19:59.733 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-22T03:19:59.746 INFO:teuthology.orchestra.run.smithi144.stdout:(22/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x 4.4 MB/s | 19 MB 00:04 2024-09-22T03:19:59.769 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-22T03:19:59.812 INFO:teuthology.orchestra.run.smithi144.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 494 kB/s | 26 kB 00:00 2024-09-22T03:19:59.838 INFO:teuthology.orchestra.run.smithi144.stdout:(24/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.0 MB/s | 308 kB 00:00 2024-09-22T03:19:59.863 INFO:teuthology.orchestra.run.smithi144.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-22T03:19:59.888 INFO:teuthology.orchestra.run.smithi144.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-22T03:19:59.930 INFO:teuthology.orchestra.run.smithi144.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-22T03:20:00.005 INFO:teuthology.orchestra.run.smithi144.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-09-22T03:20:00.089 INFO:teuthology.orchestra.run.smithi144.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-22T03:20:00.248 INFO:teuthology.orchestra.run.smithi144.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-22T03:20:00.266 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-22T03:20:00.297 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:20:00.373 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:20:00.446 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-22T03:20:00.474 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-22T03:20:01.048 INFO:teuthology.orchestra.run.smithi144.stdout:(31/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 3.1 MB/s | 3.2 MB 00:01 2024-09-22T03:20:01.165 INFO:teuthology.orchestra.run.smithi144.stdout:(32/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 2.9 MB/s | 3.1 MB 00:01 2024-09-22T03:20:01.169 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:20:01.169 INFO:teuthology.orchestra.run.smithi144.stdout:Total 9.5 MB/s | 58 MB 00:06 2024-09-22T03:20:01.267 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:20:01.303 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:20:01.303 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:20:01.970 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:20:01.971 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:20:02.688 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:20:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:20:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:20:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:20:03.433 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:20:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-22T03:20:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-22T03:20:03.433 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:20:03.467 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:20:03.860 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-22T03:20:03.951 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:20:04.045 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:20:04.142 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:20:04.177 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:20:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:20:04.264 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-09-22T03:20:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-09-22T03:20:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:20:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 7/34 2024-09-22T03:20:04.567 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:20:04.603 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 8/34 2024-09-22T03:20:04.639 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-09-22T03:20:04.684 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 10/34 2024-09-22T03:20:04.712 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:20:04.797 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 12/34 2024-09-22T03:20:04.825 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:20:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:20:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-09-22T03:20:05.054 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-22T03:20:05.092 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-22T03:20:05.146 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-22T03:20:05.182 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-22T03:20:05.212 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-22T03:20:05.245 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-22T03:20:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-22T03:20:05.632 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:20:05.723 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-22T03:20:05.753 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-22T03:20:05.789 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-22T03:20:06.271 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-22T03:20:06.301 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:20:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-22T03:20:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-22T03:20:06.487 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-22T03:20:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:20:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:20:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-22T03:20:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:20:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-22T03:20:09.425 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-22T03:20:09.425 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-22T03:20:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-09-22T03:20:13.535 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:14.525 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:20:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:20:14.552 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:20:14.552 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-22T03:20:14.553 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:20:14.553 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:20:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:20:14.644 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:20:14.645 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-22T03:20:15.801 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-22T03:20:15.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-22T03:20:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-22T03:20:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-22T03:20:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-22T03:20:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-22T03:20:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-22T03:20:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-22T03:20:15.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-22T03:20:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:20:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-22T03:20:15.804 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-22T03:20:16.852 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-22T03:20:16.853 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.854 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-20.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:16.855 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:20:17.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-test 2024-09-22T03:20:17.780 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:27 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:20:17.916 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 124 M 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: libasan x86_64 11.5.0-2.el9 appstream 414 k 2024-09-22T03:20:17.917 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 158 k 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout:Install 7 Packages 2024-09-22T03:20:17.918 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:17.919 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 125 M 2024-09-22T03:20:17.919 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.1 G 2024-09-22T03:20:17.919 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:20:18.242 INFO:teuthology.orchestra.run.smithi129.stdout:(1/7): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 729 kB/s | 158 kB 00:00 2024-09-22T03:20:18.268 INFO:teuthology.orchestra.run.smithi129.stdout:(2/7): libasan-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 414 kB 00:00 2024-09-22T03:20:18.351 INFO:teuthology.orchestra.run.smithi129.stdout:(3/7): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-22T03:20:18.410 INFO:teuthology.orchestra.run.smithi129.stdout:(4/7): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-22T03:20:18.460 INFO:teuthology.orchestra.run.smithi129.stdout:(5/7): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.4 MB/s | 122 kB 00:00 2024-09-22T03:20:18.485 INFO:teuthology.orchestra.run.smithi129.stdout:(6/7): libubsan-11.5.0-2.el9.x86_64.rpm 754 kB/s | 183 kB 00:00 2024-09-22T03:20:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:20:19.329 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-09-22T03:20:19.329 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-09-22T03:20:19.330 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-09-22T03:20:19.330 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-09-22T03:20:19.330 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-09-22T03:20:19.330 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-09-22T03:20:19.330 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-09-22T03:20:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:20.333 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:20:20.359 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-22T03:20:20.359 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:20:20.359 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-22T03:20:20.359 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:20:20.359 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:20:20.359 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:20.361 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:20:20.452 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-22T03:20:20.452 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-22T03:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-22T03:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-22T03:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-22T03:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-22T03:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-22T03:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-22T03:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-22T03:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-22T03:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-22T03:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-22T03:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-22T03:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-22T03:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-22T03:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-22T03:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-22T03:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-22T03:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-22T03:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-22T03:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-22T03:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-22T03:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-22T03:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-22T03:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-22T03:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-22T03:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-22T03:20:21.835 INFO:teuthology.orchestra.run.smithi129.stdout:(7/7): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 33 MB/s | 124 MB 00:03 2024-09-22T03:20:21.838 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:20:21.838 INFO:teuthology.orchestra.run.smithi129.stdout:Total 32 MB/s | 125 MB 00:03 2024-09-22T03:20:21.892 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:20:21.912 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:20:21.912 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:20:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-22T03:20:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:22.695 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-09-22T03:20:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.695 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-20.el9.x86_64 2024-09-22T03:20:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-22T03:20:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:22.698 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:20:22.734 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:20:22.734 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:20:23.052 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-test 2024-09-22T03:20:23.631 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:31 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:20:23.773 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 124 M 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: libasan x86_64 11.5.0-2.el9 appstream 414 k 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 158 k 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-22T03:20:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-22T03:20:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-22T03:20:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:23.775 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:20:23.775 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:20:23.775 INFO:teuthology.orchestra.run.smithi144.stdout:Install 7 Packages 2024-09-22T03:20:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:23.775 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 125 M 2024-09-22T03:20:23.776 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 1.1 G 2024-09-22T03:20:23.776 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:20:23.788 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:20:23.875 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/7 2024-09-22T03:20:23.921 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/7 2024-09-22T03:20:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/7 2024-09-22T03:20:23.995 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:20:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libasan-11.5.0-2.el9.x86_64 5/7 2024-09-22T03:20:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:20:24.090 INFO:teuthology.orchestra.run.smithi144.stdout:(1/7): libasan-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 414 kB 00:00 2024-09-22T03:20:24.149 INFO:teuthology.orchestra.run.smithi144.stdout:(2/7): libubsan-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 183 kB 00:00 2024-09-22T03:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:(3/7): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-22T03:20:24.258 INFO:teuthology.orchestra.run.smithi144.stdout:(4/7): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-22T03:20:24.283 INFO:teuthology.orchestra.run.smithi144.stdout:(5/7): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 394 kB/s | 158 kB 00:00 2024-09-22T03:20:24.333 INFO:teuthology.orchestra.run.smithi144.stdout:(6/7): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.6 MB/s | 122 kB 00:00 2024-09-22T03:20:28.933 INFO:teuthology.orchestra.run.smithi144.stdout:(7/7): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 25 MB/s | 124 MB 00:05 2024-09-22T03:20:28.937 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:20:28.937 INFO:teuthology.orchestra.run.smithi144.stdout:Total 24 MB/s | 125 MB 00:05 2024-09-22T03:20:28.991 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:20:29.012 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:20:29.012 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:20:29.835 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:20:29.835 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:20:30.892 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:20:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/7 2024-09-22T03:20:31.001 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/7 2024-09-22T03:20:31.043 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/7 2024-09-22T03:20:31.084 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:20:31.125 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libasan-11.5.0-2.el9.x86_64 5/7 2024-09-22T03:20:31.153 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:20:41.228 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:20:41.303 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:20:48.226 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/7 2024-09-22T03:20:48.299 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:20:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:20:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/7 2024-09-22T03:20:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/7 2024-09-22T03:20:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libasan-11.5.0-2.el9.x86_64 3/7 2024-09-22T03:20:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:20:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 5/7 2024-09-22T03:20:49.629 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/7 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 7/7 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: libasan-11.5.0-2.el9.x86_64 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-22T03:20:49.966 INFO:teuthology.orchestra.run.smithi129.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-22T03:20:49.967 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:49.967 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:20:50.275 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph 2024-09-22T03:20:50.854 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:00 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:20:50.990 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.3 M 2024-09-22T03:20:50.993 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.9 M 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 120 M 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-22T03:20:50.994 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-22T03:20:50.995 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout:Install 38 Packages 2024-09-22T03:20:50.996 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:20:50.997 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 134 M 2024-09-22T03:20:50.997 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.4 G 2024-09-22T03:20:50.997 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:20:51.906 INFO:teuthology.orchestra.run.smithi129.stdout:(1/38): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 47 kB/s | 6.6 kB 00:00 2024-09-22T03:20:52.106 INFO:teuthology.orchestra.run.smithi129.stdout:(2/38): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 4.5 MB/s | 1.5 MB 00:00 2024-09-22T03:20:52.157 INFO:teuthology.orchestra.run.smithi129.stdout:(3/38): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 5.9 MB/s | 2.3 MB 00:00 2024-09-22T03:20:52.224 INFO:teuthology.orchestra.run.smithi129.stdout:(4/38): ceph-mgr-modules-core-19.1.1-373.g4e45e 3.7 MB/s | 249 kB 00:00 2024-09-22T03:20:52.341 INFO:teuthology.orchestra.run.smithi129.stdout:(5/38): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-22T03:20:52.432 INFO:teuthology.orchestra.run.smithi129.stdout:(6/38): python3-cryptography-36.0.1-4.el9.x86_6 13 MB/s | 1.2 MB 00:00 2024-09-22T03:20:52.474 INFO:teuthology.orchestra.run.smithi129.stdout:(7/38): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-22T03:20:52.516 INFO:teuthology.orchestra.run.smithi129.stdout:(8/38): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-22T03:20:52.559 INFO:teuthology.orchestra.run.smithi129.stdout:(9/38): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-09-22T03:20:52.601 INFO:teuthology.orchestra.run.smithi129.stdout:(10/38): python3-urllib3-1.26.5-6.el9.noarch.rp 5.1 MB/s | 215 kB 00:00 2024-09-22T03:20:52.726 INFO:teuthology.orchestra.run.smithi129.stdout:(11/38): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-22T03:20:52.829 INFO:teuthology.orchestra.run.smithi129.stdout:(12/38): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_ 5.3 MB/s | 4.9 MB 00:00 2024-09-22T03:20:52.863 INFO:teuthology.orchestra.run.smithi129.stdout:(13/38): protobuf-3.14.0-13.el9.x86_64.rpm 7.4 MB/s | 1.0 MB 00:00 2024-09-22T03:20:52.905 INFO:teuthology.orchestra.run.smithi129.stdout:(14/38): python3-markupsafe-1.1.1-12.el9.x86_64 838 kB/s | 35 kB 00:00 2024-09-22T03:20:52.947 INFO:teuthology.orchestra.run.smithi129.stdout:(15/38): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-22T03:20:52.972 INFO:teuthology.orchestra.run.smithi129.stdout:(16/38): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-22T03:20:53.006 INFO:teuthology.orchestra.run.smithi129.stdout:(17/38): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-22T03:20:53.031 INFO:teuthology.orchestra.run.smithi129.stdout:(18/38): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-22T03:20:53.090 INFO:teuthology.orchestra.run.smithi129.stdout:(19/38): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-22T03:20:53.115 INFO:teuthology.orchestra.run.smithi129.stdout:(20/38): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-22T03:20:53.141 INFO:teuthology.orchestra.run.smithi129.stdout:(21/38): lua-devel-5.4.4-4.el9.x86_64.rpm 166 kB/s | 22 kB 00:00 2024-09-22T03:20:53.183 INFO:teuthology.orchestra.run.smithi129.stdout:(22/38): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-22T03:20:53.208 INFO:teuthology.orchestra.run.smithi129.stdout:(23/38): python3-jaraco-8.2.1-3.el9.noarch.rpm 465 kB/s | 11 kB 00:00 2024-09-22T03:20:53.233 INFO:teuthology.orchestra.run.smithi129.stdout:(24/38): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-22T03:20:53.267 INFO:teuthology.orchestra.run.smithi129.stdout:(25/38): python3-jaraco-collections-3.0.0-8.el9 694 kB/s | 23 kB 00:00 2024-09-22T03:20:53.292 INFO:teuthology.orchestra.run.smithi129.stdout:(26/38): python3-cherrypy-18.6.1-2.el9.noarch.r 2.3 MB/s | 358 kB 00:00 2024-09-22T03:20:53.318 INFO:teuthology.orchestra.run.smithi129.stdout:(27/38): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-22T03:20:53.343 INFO:teuthology.orchestra.run.smithi129.stdout:(28/38): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-22T03:20:53.368 INFO:teuthology.orchestra.run.smithi129.stdout:(29/38): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-22T03:20:53.393 INFO:teuthology.orchestra.run.smithi129.stdout:(30/38): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-22T03:20:53.419 INFO:teuthology.orchestra.run.smithi129.stdout:(31/38): python3-portend-3.1.0-2.el9.noarch.rpm 658 kB/s | 16 kB 00:00 2024-09-22T03:20:53.444 INFO:teuthology.orchestra.run.smithi129.stdout:(32/38): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-22T03:20:53.469 INFO:teuthology.orchestra.run.smithi129.stdout:(33/38): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-22T03:20:53.495 INFO:teuthology.orchestra.run.smithi129.stdout:(34/38): python3-tempora-5.0.0-2.el9.noarch.rpm 716 kB/s | 36 kB 00:00 2024-09-22T03:20:53.520 INFO:teuthology.orchestra.run.smithi129.stdout:(35/38): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-22T03:20:53.537 INFO:teuthology.orchestra.run.smithi129.stdout:(36/38): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-22T03:20:53.571 INFO:teuthology.orchestra.run.smithi129.stdout:(37/38): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-09-22T03:20:55.888 INFO:teuthology.orchestra.run.smithi129.stdout:(38/38): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_ 32 MB/s | 120 MB 00:03 2024-09-22T03:20:55.893 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:20:55.893 INFO:teuthology.orchestra.run.smithi129.stdout:Total 27 MB/s | 134 MB 00:04 2024-09-22T03:20:56.064 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:20:56.105 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:20:56.105 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:20:56.849 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/7 2024-09-22T03:20:56.850 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/7 2024-09-22T03:20:56.850 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/7 2024-09-22T03:20:56.850 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libasan-11.5.0-2.el9.x86_64 3/7 2024-09-22T03:20:56.850 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 4/7 2024-09-22T03:20:56.850 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 5/7 2024-09-22T03:20:56.850 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/7 2024-09-22T03:20:57.167 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:20:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:20:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 7/7 2024-09-22T03:20:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:57.180 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:20:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: libasan-11.5.0-2.el9.x86_64 2024-09-22T03:20:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:20:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-22T03:20:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-22T03:20:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-22T03:20:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-22T03:20:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:57.181 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:20:57.490 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph 2024-09-22T03:20:58.075 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:06 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:20:58.210 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.3 M 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.9 M 2024-09-22T03:20:58.213 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 120 M 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-22T03:20:58.214 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-22T03:20:58.215 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout:Install 38 Packages 2024-09-22T03:20:58.216 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:20:58.217 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 134 M 2024-09-22T03:20:58.217 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 1.4 G 2024-09-22T03:20:58.217 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:20:58.680 INFO:teuthology.orchestra.run.smithi144.stdout:(1/38): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 45 kB/s | 6.6 kB 00:00 2024-09-22T03:20:58.855 INFO:teuthology.orchestra.run.smithi144.stdout:(2/38): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-22T03:20:58.905 INFO:teuthology.orchestra.run.smithi144.stdout:(3/38): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 6.1 MB/s | 2.3 MB 00:00 2024-09-22T03:20:58.972 INFO:teuthology.orchestra.run.smithi144.stdout:(4/38): ceph-mgr-modules-core-19.1.1-373.g4e45e 3.7 MB/s | 249 kB 00:00 2024-09-22T03:20:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:20:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/38 2024-09-22T03:20:59.239 INFO:teuthology.orchestra.run.smithi144.stdout:(5/38): python3-cffi-1.14.5-5.el9.x86_64.rpm 950 kB/s | 253 kB 00:00 2024-09-22T03:20:59.240 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/38 2024-09-22T03:20:59.267 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/38 2024-09-22T03:20:59.338 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/38 2024-09-22T03:20:59.392 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/38 2024-09-22T03:20:59.425 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/38 2024-09-22T03:20:59.456 INFO:teuthology.orchestra.run.smithi144.stdout:(6/38): python3-cryptography-36.0.1-4.el9.x86_6 5.7 MB/s | 1.2 MB 00:00 2024-09-22T03:20:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/38 2024-09-22T03:20:59.531 INFO:teuthology.orchestra.run.smithi144.stdout:(7/38): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-22T03:20:59.587 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/38 2024-09-22T03:20:59.628 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/38 2024-09-22T03:20:59.640 INFO:teuthology.orchestra.run.smithi144.stdout:(8/38): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_6 5.1 MB/s | 4.9 MB 00:00 2024-09-22T03:20:59.663 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/38 2024-09-22T03:20:59.673 INFO:teuthology.orchestra.run.smithi144.stdout:(9/38): python3-pycparser-2.20-6.el9.noarch.rpm 952 kB/s | 135 kB 00:00 2024-09-22T03:20:59.693 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/38 2024-09-22T03:20:59.727 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/38 2024-09-22T03:20:59.760 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/38 2024-09-22T03:20:59.782 INFO:teuthology.orchestra.run.smithi144.stdout:(10/38): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-09-22T03:20:59.809 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/38 2024-09-22T03:20:59.845 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/38 2024-09-22T03:20:59.867 INFO:teuthology.orchestra.run.smithi144.stdout:(11/38): python3-requests-2.25.1-8.el9.noarch.r 553 kB/s | 125 kB 00:00 2024-09-22T03:20:59.879 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/38 2024-09-22T03:20:59.927 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/38 2024-09-22T03:20:59.964 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/38 2024-09-22T03:21:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/38 2024-09-22T03:21:00.184 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/38 2024-09-22T03:21:00.236 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/38 2024-09-22T03:21:00.325 INFO:teuthology.orchestra.run.smithi144.stdout:(12/38): lua-5.4.4-4.el9.x86_64.rpm 348 kB/s | 188 kB 00:00 2024-09-22T03:21:00.533 INFO:teuthology.orchestra.run.smithi144.stdout:(13/38): python3-mako-1.1.4-6.el9.noarch.rpm 827 kB/s | 172 kB 00:00 2024-09-22T03:21:00.650 INFO:teuthology.orchestra.run.smithi144.stdout:(14/38): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-09-22T03:21:00.692 INFO:teuthology.orchestra.run.smithi144.stdout:(15/38): protobuf-3.14.0-13.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2024-09-22T03:21:00.759 INFO:teuthology.orchestra.run.smithi144.stdout:(16/38): python3-packaging-20.9-5.el9.noarch.rp 712 kB/s | 77 kB 00:00 2024-09-22T03:21:00.809 INFO:teuthology.orchestra.run.smithi144.stdout:(17/38): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-09-22T03:21:00.868 INFO:teuthology.orchestra.run.smithi144.stdout:(18/38): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-22T03:21:00.918 INFO:teuthology.orchestra.run.smithi144.stdout:(19/38): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-22T03:21:00.952 INFO:teuthology.orchestra.run.smithi144.stdout:(20/38): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-22T03:21:00.986 INFO:teuthology.orchestra.run.smithi144.stdout:(21/38): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 649 kB/s | 43 kB 00:00 2024-09-22T03:21:01.011 INFO:teuthology.orchestra.run.smithi144.stdout:(22/38): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.0 MB/s | 172 kB 00:00 2024-09-22T03:21:01.036 INFO:teuthology.orchestra.run.smithi144.stdout:(23/38): python3-jaraco-8.2.1-3.el9.noarch.rpm 428 kB/s | 11 kB 00:00 2024-09-22T03:21:01.053 INFO:teuthology.orchestra.run.smithi144.stdout:(24/38): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-22T03:21:01.070 INFO:teuthology.orchestra.run.smithi144.stdout:(25/38): python3-jaraco-collections-3.0.0-8.el9 1.4 MB/s | 23 kB 00:00 2024-09-22T03:21:01.095 INFO:teuthology.orchestra.run.smithi144.stdout:(26/38): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-09-22T03:21:01.112 INFO:teuthology.orchestra.run.smithi144.stdout:(27/38): python3-jaraco-functools-3.5.0-2.el9.n 460 kB/s | 19 kB 00:00 2024-09-22T03:21:01.129 INFO:teuthology.orchestra.run.smithi144.stdout:(28/38): python3-jaraco-text-3.2.0-6.el9.noarch 588 kB/s | 20 kB 00:00 2024-09-22T03:21:01.146 INFO:teuthology.orchestra.run.smithi144.stdout:(29/38): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-22T03:21:01.172 INFO:teuthology.orchestra.run.smithi144.stdout:(30/38): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-22T03:21:01.189 INFO:teuthology.orchestra.run.smithi144.stdout:(31/38): python3-portend-3.1.0-2.el9.noarch.rpm 986 kB/s | 16 kB 00:00 2024-09-22T03:21:01.214 INFO:teuthology.orchestra.run.smithi144.stdout:(32/38): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-22T03:21:01.247 INFO:teuthology.orchestra.run.smithi144.stdout:(33/38): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-22T03:21:01.264 INFO:teuthology.orchestra.run.smithi144.stdout:(34/38): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-22T03:21:01.298 INFO:teuthology.orchestra.run.smithi144.stdout:(35/38): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-22T03:21:01.315 INFO:teuthology.orchestra.run.smithi144.stdout:(36/38): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-22T03:21:01.349 INFO:teuthology.orchestra.run.smithi144.stdout:(37/38): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-09-22T03:21:02.416 INFO:teuthology.orchestra.run.smithi144.stdout:(38/38): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_ 34 MB/s | 120 MB 00:03 2024-09-22T03:21:02.420 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:02.420 INFO:teuthology.orchestra.run.smithi144.stdout:Total 32 MB/s | 134 MB 00:04 2024-09-22T03:21:02.588 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:21:02.627 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:21:02.627 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:21:03.738 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:21:03.738 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:21:05.731 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:21:05.793 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/38 2024-09-22T03:21:05.833 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/38 2024-09-22T03:21:05.862 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/38 2024-09-22T03:21:05.934 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/38 2024-09-22T03:21:05.978 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/38 2024-09-22T03:21:06.010 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/38 2024-09-22T03:21:06.102 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/38 2024-09-22T03:21:06.174 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/38 2024-09-22T03:21:06.221 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/38 2024-09-22T03:21:06.259 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/38 2024-09-22T03:21:06.288 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/38 2024-09-22T03:21:06.323 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/38 2024-09-22T03:21:06.356 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/38 2024-09-22T03:21:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/38 2024-09-22T03:21:06.557 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/38 2024-09-22T03:21:06.591 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/38 2024-09-22T03:21:06.640 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/38 2024-09-22T03:21:06.671 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/38 2024-09-22T03:21:06.747 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/38 2024-09-22T03:21:06.885 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/38 2024-09-22T03:21:06.938 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/38 2024-09-22T03:21:19.045 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/38 2024-09-22T03:21:19.076 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:21:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:21:19.105 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:19.105 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-22T03:21:19.105 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:21:19.105 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:21:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/38 2024-09-22T03:21:19.239 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/38 2024-09-22T03:21:19.288 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 26/38 2024-09-22T03:21:19.343 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 27/38 2024-09-22T03:21:19.528 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 28/38 2024-09-22T03:21:19.585 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 29/38 2024-09-22T03:21:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/38 2024-09-22T03:21:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 31/38 2024-09-22T03:21:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 32/38 2024-09-22T03:21:19.901 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 33/38 2024-09-22T03:21:20.061 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 34/38 2024-09-22T03:21:20.096 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:21:20.120 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:21:20.120 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:20.120 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-22T03:21:20.120 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:21:20.121 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:21:20.121 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:20.589 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:21:20.614 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:21:20.614 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:20.614 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-22T03:21:20.614 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:21:20.614 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:21:20.614 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:21:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:21:20.887 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:20.887 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-22T03:21:20.887 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:21:20.887 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:21:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:20.947 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:21:25.768 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/38 2024-09-22T03:21:25.792 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:21:25.821 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/38 2024-09-22T03:21:25.821 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:25.821 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-22T03:21:25.821 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:21:25.822 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:21:25.822 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/38 2024-09-22T03:21:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/38 2024-09-22T03:21:26.013 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 26/38 2024-09-22T03:21:26.077 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 27/38 2024-09-22T03:21:26.263 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 28/38 2024-09-22T03:21:26.310 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 29/38 2024-09-22T03:21:26.388 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/38 2024-09-22T03:21:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 31/38 2024-09-22T03:21:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 32/38 2024-09-22T03:21:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 33/38 2024-09-22T03:21:26.827 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 34/38 2024-09-22T03:21:26.853 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:21:26.883 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/38 2024-09-22T03:21:26.883 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:26.883 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-22T03:21:26.883 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:21:26.883 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:21:26.883 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:21:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/38 2024-09-22T03:21:27.369 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:27.369 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-22T03:21:27.369 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:21:27.369 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:21:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:27.596 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:21:27.620 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/38 2024-09-22T03:21:27.620 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:27.620 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-22T03:21:27.620 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:21:27.620 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:21:27.620 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/38 2024-09-22T03:21:31.182 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 14/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 15/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 16/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 17/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 18/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 19/38 2024-09-22T03:21:31.183 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 20/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 21/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 22/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 23/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 24/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 25/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 26/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 27/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 28/38 2024-09-22T03:21:31.184 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 29/38 2024-09-22T03:21:31.185 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 30/38 2024-09-22T03:21:31.185 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 31/38 2024-09-22T03:21:31.185 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 32/38 2024-09-22T03:21:31.185 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 33/38 2024-09-22T03:21:31.185 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 34/38 2024-09-22T03:21:31.186 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 35/38 2024-09-22T03:21:31.186 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 36/38 2024-09-22T03:21:31.186 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 37/38 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 38/38 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:32.111 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-22T03:21:32.112 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-22T03:21:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:32.114 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:21:32.396 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-base 2024-09-22T03:21:32.977 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:42 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:21:33.066 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:21:33.108 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:21:33.109 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:21:33.109 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:21:33.172 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install cephadm 2024-09-22T03:21:33.751 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:43 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:21:33.883 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 765 k 2024-09-22T03:21:33.884 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 771 k 2024-09-22T03:21:33.885 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:21:34.149 INFO:teuthology.orchestra.run.smithi129.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-22T03:21:34.149 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:34.150 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-22T03:21:34.150 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:21:34.155 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:21:34.155 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:21:34.161 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:21:34.161 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:21:34.233 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:21:34.692 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:34.770 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:35.137 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:35.390 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:35.390 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:35.390 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:21:35.390 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:35.390 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:35.390 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:21:35.606 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-immutable-object-cache 2024-09-22T03:21:36.201 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:46 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:21:36.337 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:21:36.337 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 147 k 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:21:36.338 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:36.339 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 147 k 2024-09-22T03:21:36.339 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 481 k 2024-09-22T03:21:36.339 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:21:36.516 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 829 kB/s | 147 kB 00:00 2024-09-22T03:21:36.517 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:36.518 INFO:teuthology.orchestra.run.smithi129.stdout:Total 822 kB/s | 147 kB 00:00 2024-09-22T03:21:36.518 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:21:36.525 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:21:36.525 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:21:36.582 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:21:36.582 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:21:36.736 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:21:36.818 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:21:36.843 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:21:36.843 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:36.843 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-22T03:21:36.843 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:37.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:21:37.785 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:37.785 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:21:37.785 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:37.786 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:37.786 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:21:38.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr 2024-09-22T03:21:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 38/38 2024-09-22T03:21:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/38 2024-09-22T03:21:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 14/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 15/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 16/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 17/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 18/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 19/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 20/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 21/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 22/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 23/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 24/38 2024-09-22T03:21:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 25/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 26/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 27/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 28/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 29/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 30/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 31/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 32/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 33/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 34/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 35/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 36/38 2024-09-22T03:21:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 37/38 2024-09-22T03:21:38.611 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:48 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:21:38.703 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:21:38.747 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:21:38.748 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:21:38.748 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:21:38.808 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-dashboard 2024-09-22T03:21:39.233 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 38/38 2024-09-22T03:21:39.233 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:39.233 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:21:39.233 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:39.233 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:39.233 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-22T03:21:39.234 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-22T03:21:39.235 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:39.236 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:21:39.391 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:49 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:21:39.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-base 2024-09-22T03:21:39.524 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:21:39.525 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:21:39.525 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:39.525 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:21:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-22T03:21:39.525 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:21:39.525 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout:Install 11 Packages 2024-09-22T03:21:39.526 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:39.528 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 7.7 M 2024-09-22T03:21:39.528 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 101 M 2024-09-22T03:21:39.528 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:21:40.080 INFO:teuthology.orchestra.run.smithi129.stdout:(1/11): ceph-grafana-dashboards-19.1.1-373.g4e4 215 kB/s | 32 kB 00:00 2024-09-22T03:21:40.104 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:48 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:21:40.105 INFO:teuthology.orchestra.run.smithi129.stdout:(2/11): ceph-prometheus-alerts-19.1.1-373.g4e45 95 kB/s | 16 kB 00:00 2024-09-22T03:21:40.192 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:21:40.234 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:21:40.235 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:21:40.235 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:21:40.310 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install cephadm 2024-09-22T03:21:40.314 INFO:teuthology.orchestra.run.smithi129.stdout:(3/11): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 9.3 MB/s | 3.6 MB 00:00 2024-09-22T03:21:40.373 INFO:teuthology.orchestra.run.smithi129.stdout:(4/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-22T03:21:40.398 INFO:teuthology.orchestra.run.smithi129.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-22T03:21:40.423 INFO:teuthology.orchestra.run.smithi129.stdout:(6/11): protobuf-compiler-3.14.0-13.el9.x86_64. 2.7 MB/s | 863 kB 00:00 2024-09-22T03:21:40.483 INFO:teuthology.orchestra.run.smithi129.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-09-22T03:21:40.507 INFO:teuthology.orchestra.run.smithi129.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.3 MB/s | 31 kB 00:00 2024-09-22T03:21:40.557 INFO:teuthology.orchestra.run.smithi129.stdout:(9/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 13 MB/s | 2.0 MB 00:00 2024-09-22T03:21:40.583 INFO:teuthology.orchestra.run.smithi129.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-22T03:21:40.742 INFO:teuthology.orchestra.run.smithi129.stdout:(11/11): python3-protobuf-3.14.0-13.el9.noarch. 407 kB/s | 269 kB 00:00 2024-09-22T03:21:40.745 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:40.745 INFO:teuthology.orchestra.run.smithi129.stdout:Total 6.3 MB/s | 7.7 MB 00:01 2024-09-22T03:21:40.775 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:21:40.792 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:21:40.792 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:21:40.883 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:48 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:21:40.991 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:21:40.991 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:21:41.014 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:21:41.014 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 765 k 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 771 k 2024-09-22T03:21:41.015 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:21:41.278 INFO:teuthology.orchestra.run.smithi144.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-22T03:21:41.278 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:41.278 INFO:teuthology.orchestra.run.smithi144.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-22T03:21:41.279 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:21:41.284 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:21:41.284 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:21:41.290 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:21:41.290 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:21:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:21:41.480 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:21:41.590 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-22T03:21:41.620 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-22T03:21:41.729 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-22T03:21:41.821 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:41.855 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-22T03:21:41.907 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-22T03:21:41.914 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:41.999 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-22T03:21:42.051 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-22T03:21:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:21:42.111 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 9/11 2024-09-22T03:21:42.283 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:42.495 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:21:42.495 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:42.496 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:21:42.496 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:42.496 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:42.496 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:21:42.738 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-immutable-object-cache 2024-09-22T03:21:43.170 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 10/11 2024-09-22T03:21:43.212 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:21:43.331 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:51 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:21:43.468 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:21:43.468 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:43.468 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-09-22T03:21:43.468 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:43.468 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:21:43.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 147 k 2024-09-22T03:21:43.468 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:43.469 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:21:43.469 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:43.469 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:21:43.469 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:43.469 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 147 k 2024-09-22T03:21:43.469 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 481 k 2024-09-22T03:21:43.469 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:21:43.674 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 718 kB/s | 147 kB 00:00 2024-09-22T03:21:43.675 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:43.675 INFO:teuthology.orchestra.run.smithi144.stdout:Total 713 kB/s | 147 kB 00:00 2024-09-22T03:21:43.675 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:21:43.683 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:21:43.683 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:21:43.740 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:21:43.740 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:21:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:21:43.959 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:21:43.984 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:21:43.984 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:21:43.984 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-22T03:21:43.984 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:21:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/11 2024-09-22T03:21:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/11 2024-09-22T03:21:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/11 2024-09-22T03:21:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-22T03:21:44.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-22T03:21:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-22T03:21:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-22T03:21:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:21:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-22T03:21:44.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-22T03:21:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-22T03:21:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-22T03:21:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-22T03:21:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-22T03:21:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-22T03:21:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:44.869 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:21:44.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-22T03:21:44.938 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:21:44.938 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:21:44.938 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:21:45.121 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-22T03:21:45.181 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr 2024-09-22T03:21:45.694 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:55 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:21:45.779 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:53 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:21:45.827 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-22T03:21:45.829 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-22T03:21:45.830 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:45.831 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 38 M 2024-09-22T03:21:45.831 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 212 M 2024-09-22T03:21:45.831 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:21:45.872 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:21:45.914 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:21:45.915 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:21:45.916 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:21:45.988 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-dashboard 2024-09-22T03:21:46.117 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 882 kB/s | 191 kB 00:00 2024-09-22T03:21:46.209 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-22T03:21:46.226 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-22T03:21:46.309 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-09-22T03:21:46.401 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): ceph-mgr-diskprediction-local-19.1.1-37 15 MB/s | 7.4 MB 00:00 2024-09-22T03:21:46.426 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-22T03:21:46.485 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-09-22T03:21:46.510 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-22T03:21:46.569 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:54 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:21:46.577 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.7 MB/s | 250 kB 00:00 2024-09-22T03:21:46.653 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-09-22T03:21:46.702 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:21:46.703 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:46.703 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-22T03:21:46.704 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-22T03:21:46.705 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-22T03:21:46.705 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:46.705 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:21:46.705 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:21:46.705 INFO:teuthology.orchestra.run.smithi144.stdout:Install 11 Packages 2024-09-22T03:21:46.705 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:46.705 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 7.7 M 2024-09-22T03:21:46.706 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 101 M 2024-09-22T03:21:46.706 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:21:46.887 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 53 MB/s | 16 MB 00:00 2024-09-22T03:21:46.970 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 8.7 MB/s | 4.9 MB 00:00 2024-09-22T03:21:46.971 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:46.971 INFO:teuthology.orchestra.run.smithi129.stdout:Total 33 MB/s | 38 MB 00:01 2024-09-22T03:21:47.020 INFO:teuthology.orchestra.run.smithi144.stdout:(1/11): ceph-grafana-dashboards-19.1.1-373.g4e4 210 kB/s | 32 kB 00:00 2024-09-22T03:21:47.045 INFO:teuthology.orchestra.run.smithi144.stdout:(2/11): ceph-prometheus-alerts-19.1.1-373.g4e45 93 kB/s | 16 kB 00:00 2024-09-22T03:21:47.220 INFO:teuthology.orchestra.run.smithi144.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-22T03:21:47.279 INFO:teuthology.orchestra.run.smithi144.stdout:(4/11): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 8.7 MB/s | 3.6 MB 00:00 2024-09-22T03:21:47.312 INFO:teuthology.orchestra.run.smithi144.stdout:(5/11): protobuf-compiler-3.14.0-13.el9.x86_64. 3.2 MB/s | 863 kB 00:00 2024-09-22T03:21:47.325 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:21:47.351 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:21:47.351 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:21:47.362 INFO:teuthology.orchestra.run.smithi144.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 449 kB/s | 19 kB 00:00 2024-09-22T03:21:47.396 INFO:teuthology.orchestra.run.smithi144.stdout:(7/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.0 MB/s | 548 kB 00:00 2024-09-22T03:21:47.421 INFO:teuthology.orchestra.run.smithi144.stdout:(8/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.5 MB/s | 144 kB 00:00 2024-09-22T03:21:47.447 INFO:teuthology.orchestra.run.smithi144.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 611 kB/s | 31 kB 00:00 2024-09-22T03:21:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-22T03:21:47.564 INFO:teuthology.orchestra.run.smithi144.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.3 MB/s | 2.0 MB 00:00 2024-09-22T03:21:47.568 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:47.568 INFO:teuthology.orchestra.run.smithi144.stdout:Total 8.9 MB/s | 7.7 MB 00:00 2024-09-22T03:21:47.598 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:21:47.616 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:21:47.616 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:21:47.774 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:21:47.774 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:21:47.826 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:21:47.827 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:21:48.318 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:21:48.403 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-22T03:21:48.442 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-22T03:21:48.534 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:21:48.552 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-22T03:21:48.604 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-22T03:21:48.643 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:21:48.677 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-22T03:21:48.729 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-22T03:21:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:21:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-22T03:21:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-22T03:21:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-22T03:21:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:21:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 9/11 2024-09-22T03:21:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-22T03:21:49.043 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-22T03:21:49.120 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-22T03:21:49.767 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-22T03:21:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-22T03:21:49.995 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 10/11 2024-09-22T03:21:50.150 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:21:51.116 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:21:51.326 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 11/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-22T03:21:51.327 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-22T03:21:51.629 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-22T03:21:51.630 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-22T03:21:51.630 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-22T03:21:51.630 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-22T03:21:51.630 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:51.630 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:21:51.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-22T03:21:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-22T03:21:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:21:52.452 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:00 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:21:52.584 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:21:52.585 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-09-22T03:21:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:21:52.585 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-09-22T03:21:52.585 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:21:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-22T03:21:52.585 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-22T03:21:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:52.587 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:21:52.587 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-09-22T03:21:52.587 INFO:teuthology.orchestra.run.smithi144.stdout:Install 12 Packages 2024-09-22T03:21:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:21:52.587 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 38 M 2024-09-22T03:21:52.587 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 212 M 2024-09-22T03:21:52.588 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:21:52.854 INFO:teuthology.orchestra.run.smithi144.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-22T03:21:52.896 INFO:teuthology.orchestra.run.smithi144.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-22T03:21:52.980 INFO:teuthology.orchestra.run.smithi144.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-22T03:21:53.022 INFO:teuthology.orchestra.run.smithi144.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-22T03:21:53.063 INFO:teuthology.orchestra.run.smithi144.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 928 kB/s | 38 kB 00:00 2024-09-22T03:21:53.197 INFO:teuthology.orchestra.run.smithi144.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-37 16 MB/s | 7.4 MB 00:00 2024-09-22T03:21:53.348 INFO:teuthology.orchestra.run.smithi144.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.6 MB/s | 3.0 MB 00:00 2024-09-22T03:21:53.456 INFO:teuthology.orchestra.run.smithi144.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-22T03:21:53.481 INFO:teuthology.orchestra.run.smithi144.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 862 kB/s | 245 kB 00:00 2024-09-22T03:21:53.515 INFO:teuthology.orchestra.run.smithi144.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.2 MB/s | 250 kB 00:00 2024-09-22T03:21:53.582 INFO:teuthology.orchestra.run.smithi144.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-09-22T03:21:54.703 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:21:54.703 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:21:54.704 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-22T03:21:54.774 INFO:teuthology.orchestra.run.smithi144.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-09-22T03:21:54.774 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:54.775 INFO:teuthology.orchestra.run.smithi144.stdout:Total 17 MB/s | 38 MB 00:02 2024-09-22T03:21:55.123 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:21:55.134 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-22T03:21:55.134 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:21:55.134 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:21:55.134 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-22T03:21:55.134 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-22T03:21:55.134 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:21:55.149 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:21:55.149 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:21:55.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-rook 2024-09-22T03:21:55.564 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:21:55.565 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:21:56.087 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:06 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:21:56.223 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:21:56.224 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:56.224 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:21:56.224 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-22T03:21:56.225 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-22T03:21:56.226 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-22T03:21:56.226 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-22T03:21:56.226 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:56.226 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:21:56.226 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:21:56.226 INFO:teuthology.orchestra.run.smithi129.stdout:Install 13 Packages 2024-09-22T03:21:56.226 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:21:56.227 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 2.2 M 2024-09-22T03:21:56.227 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 26 M 2024-09-22T03:21:56.227 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:21:56.333 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:21:56.413 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-22T03:21:56.455 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:21:56.552 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:21:56.604 INFO:teuthology.orchestra.run.smithi129.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 182 kB/s | 19 kB 00:00 2024-09-22T03:21:56.606 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-22T03:21:56.650 INFO:teuthology.orchestra.run.smithi129.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 169 kB/s | 26 kB 00:00 2024-09-22T03:21:56.705 INFO:teuthology.orchestra.run.smithi129.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-22T03:21:56.746 INFO:teuthology.orchestra.run.smithi129.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-09-22T03:21:56.772 INFO:teuthology.orchestra.run.smithi129.stdout:(5/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 180 kB/s | 49 kB 00:00 2024-09-22T03:21:56.791 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-22T03:21:56.797 INFO:teuthology.orchestra.run.smithi129.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.0 MB/s | 279 kB 00:00 2024-09-22T03:21:56.822 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-22T03:21:56.831 INFO:teuthology.orchestra.run.smithi129.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 639 kB/s | 54 kB 00:00 2024-09-22T03:21:56.864 INFO:teuthology.orchestra.run.smithi129.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-22T03:21:56.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-22T03:21:56.898 INFO:teuthology.orchestra.run.smithi129.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 150 kB/s | 14 kB 00:00 2024-09-22T03:21:56.931 INFO:teuthology.orchestra.run.smithi129.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-22T03:21:56.957 INFO:teuthology.orchestra.run.smithi129.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-22T03:21:56.982 INFO:teuthology.orchestra.run.smithi129.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.5 MB/s | 220 kB 00:00 2024-09-22T03:21:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-22T03:21:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:21:57.018 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.8 MB/s | 2.2 MB 00:00 2024-09-22T03:21:57.084 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:21:57.099 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:21:57.100 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:21:57.189 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:21:57.190 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:21:57.557 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-22T03:21:57.677 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-22T03:21:57.708 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:21:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-22T03:21:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-22T03:21:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-22T03:21:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-22T03:21:58.076 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-22T03:21:58.178 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-22T03:21:58.279 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-22T03:21:58.337 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-22T03:21:58.926 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:21:59.046 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-22T03:21:59.092 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-22T03:21:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-22T03:21:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-22T03:21:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:21:59.932 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-22T03:21:59.933 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-22T03:21:59.975 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-22T03:22:00.004 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:22:00.269 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-22T03:22:00.269 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-22T03:22:00.270 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-22T03:22:00.271 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-22T03:22:00.271 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:00.271 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:00.516 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-cephadm 2024-09-22T03:22:01.099 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:11 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:01.233 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-22T03:22:01.234 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout:Install 6 Packages 2024-09-22T03:22:01.235 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:01.236 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 7.0 M 2024-09-22T03:22:01.236 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 33 M 2024-09-22T03:22:01.236 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:01.861 INFO:teuthology.orchestra.run.smithi129.stdout:(1/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 747 kB/s | 146 kB 00:00 2024-09-22T03:22:01.919 INFO:teuthology.orchestra.run.smithi129.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-22T03:22:01.945 INFO:teuthology.orchestra.run.smithi129.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-22T03:22:01.978 INFO:teuthology.orchestra.run.smithi129.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-22T03:22:02.054 INFO:teuthology.orchestra.run.smithi129.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 639 kB/s | 248 kB 00:00 2024-09-22T03:22:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-22T03:22:02.551 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-22T03:22:02.552 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:22:02.552 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-22T03:22:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:03.031 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:03.463 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-rook 2024-09-22T03:22:03.779 INFO:teuthology.orchestra.run.smithi129.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.8 MB/s | 6.0 MB 00:02 2024-09-22T03:22:03.780 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:03.781 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.8 MB/s | 7.0 MB 00:02 2024-09-22T03:22:03.855 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:03.872 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:03.872 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:03.980 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:03.981 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:04.061 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:12 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:04.195 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:04.196 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:22:04.196 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:04.197 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-22T03:22:04.197 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:22:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-22T03:22:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-22T03:22:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-22T03:22:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-22T03:22:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-22T03:22:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-22T03:22:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-22T03:22:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-22T03:22:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-22T03:22:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-22T03:22:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-22T03:22:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-22T03:22:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:04.199 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:04.199 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:04.199 INFO:teuthology.orchestra.run.smithi144.stdout:Install 13 Packages 2024-09-22T03:22:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:04.201 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 2.2 M 2024-09-22T03:22:04.201 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 26 M 2024-09-22T03:22:04.201 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:04.290 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-22T03:22:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-22T03:22:04.530 INFO:teuthology.orchestra.run.smithi144.stdout:(1/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 293 kB/s | 49 kB 00:00 2024-09-22T03:22:04.872 INFO:teuthology.orchestra.run.smithi144.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 38 kB/s | 19 kB 00:00 2024-09-22T03:22:04.964 INFO:teuthology.orchestra.run.smithi144.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 43 kB/s | 26 kB 00:00 2024-09-22T03:22:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-22T03:22:05.117 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-22T03:22:05.181 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-22T03:22:05.210 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:22:05.281 INFO:teuthology.orchestra.run.smithi144.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 296 kB/s | 222 kB 00:00 2024-09-22T03:22:05.665 INFO:teuthology.orchestra.run.smithi144.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 200 kB/s | 159 kB 00:00 2024-09-22T03:22:05.698 INFO:teuthology.orchestra.run.smithi144.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 983 kB/s | 32 kB 00:00 2024-09-22T03:22:05.724 INFO:teuthology.orchestra.run.smithi144.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 367 kB/s | 279 kB 00:00 2024-09-22T03:22:05.749 INFO:teuthology.orchestra.run.smithi144.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-22T03:22:05.774 INFO:teuthology.orchestra.run.smithi144.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.3 MB/s | 220 kB 00:00 2024-09-22T03:22:05.808 INFO:teuthology.orchestra.run.smithi144.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-22T03:22:05.833 INFO:teuthology.orchestra.run.smithi144.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-22T03:22:05.892 INFO:teuthology.orchestra.run.smithi144.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-22T03:22:05.942 INFO:teuthology.orchestra.run.smithi144.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 81 kB/s | 54 kB 00:00 2024-09-22T03:22:05.945 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:05.946 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.3 MB/s | 2.2 MB 00:01 2024-09-22T03:22:05.988 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:22:05.988 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-22T03:22:05.988 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-22T03:22:05.988 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-22T03:22:05.988 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-22T03:22:05.988 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-22T03:22:06.016 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:06.034 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:06.034 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-22T03:22:06.255 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-22T03:22:06.256 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-22T03:22:06.256 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:06.256 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:06.510 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-fuse 2024-09-22T03:22:06.624 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:06.791 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-22T03:22:06.843 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-22T03:22:06.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-22T03:22:06.935 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-22T03:22:06.977 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-22T03:22:07.079 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-22T03:22:07.094 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:17 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-22T03:22:07.230 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:07.230 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:07.230 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:22:07.230 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:07.230 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 868 k 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout:Install 2 Packages 2024-09-22T03:22:07.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:07.232 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 948 k 2024-09-22T03:22:07.232 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 3.1 M 2024-09-22T03:22:07.232 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:07.245 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-22T03:22:07.423 INFO:teuthology.orchestra.run.smithi129.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 559 kB/s | 80 kB 00:00 2024-09-22T03:22:07.548 INFO:teuthology.orchestra.run.smithi129.stdout:(2/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 3.2 MB/s | 868 kB 00:00 2024-09-22T03:22:07.549 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:07.549 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.9 MB/s | 948 kB 00:00 2024-09-22T03:22:07.560 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:07.583 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:07.583 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:07.664 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:07.665 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:07.849 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:07.958 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-22T03:22:07.976 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-22T03:22:08.010 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-22T03:22:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-22T03:22:08.047 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:22:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-22T03:22:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:22:08.781 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:22:08.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-22T03:22:08.910 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-22T03:22:08.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-22T03:22:08.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-22T03:22:08.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-22T03:22:08.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-22T03:22:08.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-22T03:22:08.911 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-22T03:22:08.911 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-22T03:22:08.911 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-22T03:22:08.911 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-22T03:22:08.911 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-22T03:22:08.911 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-22T03:22:08.911 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-22T03:22:09.065 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-22T03:22:09.065 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:09.065 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:09.065 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-22T03:22:09.065 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:09.065 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:09.266 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-22T03:22:09.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-22T03:22:09.268 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-22T03:22:09.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:09.268 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:09.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-volume 2024-09-22T03:22:09.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-cephadm 2024-09-22T03:22:09.936 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:19 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:10.071 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 285 k 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:10.072 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:10.073 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:22:10.073 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:10.073 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 285 k 2024-09-22T03:22:10.073 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.3 M 2024-09-22T03:22:10.073 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:10.114 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:18 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:10.248 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:10.248 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-22T03:22:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:10.250 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:10.250 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:10.250 INFO:teuthology.orchestra.run.smithi144.stdout:Install 6 Packages 2024-09-22T03:22:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:10.250 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 7.0 M 2024-09-22T03:22:10.250 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 33 M 2024-09-22T03:22:10.251 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:10.312 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-volume-19.1.1-373.g4e45e6b4.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-22T03:22:10.313 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:10.313 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-22T03:22:10.314 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:10.322 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:10.322 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:10.354 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:10.355 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:10.527 INFO:teuthology.orchestra.run.smithi144.stdout:(1/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 733 kB/s | 146 kB 00:00 2024-09-22T03:22:10.574 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:10.611 INFO:teuthology.orchestra.run.smithi144.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-22T03:22:10.636 INFO:teuthology.orchestra.run.smithi144.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-22T03:22:10.639 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:22:10.656 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:22:10.656 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:22:10.656 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-22T03:22:10.656 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:10.661 INFO:teuthology.orchestra.run.smithi144.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-22T03:22:10.961 INFO:teuthology.orchestra.run.smithi144.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 392 kB/s | 248 kB 00:00 2024-09-22T03:22:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-22T03:22:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:Total 5.3 MB/s | 7.0 MB 00:01 2024-09-22T03:22:11.651 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:11.662 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:11.662 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:22:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:11.706 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:22:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:11.706 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:11.763 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:11.763 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:11.996 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados-devel 2024-09-22T03:22:12.101 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:12.277 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-22T03:22:12.322 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-22T03:22:12.595 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:22 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:12.730 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:12.730 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:12.730 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:22:12.731 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:12.732 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 127 k 2024-09-22T03:22:12.732 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 457 k 2024-09-22T03:22:12.732 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-22T03:22:12.933 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-22T03:22:12.943 INFO:teuthology.orchestra.run.smithi129.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 599 kB/s | 127 kB 00:00 2024-09-22T03:22:12.944 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:12.944 INFO:teuthology.orchestra.run.smithi129.stdout:Total 595 kB/s | 127 kB 00:00 2024-09-22T03:22:12.944 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:12.950 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:12.950 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:13.005 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-22T03:22:13.015 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:13.016 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:13.034 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:22:13.164 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:13.299 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-22T03:22:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-22T03:22:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-22T03:22:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-22T03:22:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-22T03:22:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-22T03:22:14.014 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:14.014 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:14.014 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:14.014 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:14.014 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:14.014 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-22T03:22:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:14.146 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:14.253 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs2 2024-09-22T03:22:14.411 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-fuse 2024-09-22T03:22:14.847 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:24 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:14.938 INFO:teuthology.orchestra.run.smithi129.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:14.984 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:14.985 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:22:14.986 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:14.994 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:22 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:15.049 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs-devel 2024-09-22T03:22:15.128 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 868 k 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-22T03:22:15.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:15.130 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:15.130 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:15.130 INFO:teuthology.orchestra.run.smithi144.stdout:Install 2 Packages 2024-09-22T03:22:15.130 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:15.130 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 948 k 2024-09-22T03:22:15.130 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 3.1 M 2024-09-22T03:22:15.130 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:15.538 INFO:teuthology.orchestra.run.smithi144.stdout:(1/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 3.0 MB/s | 868 kB 00:00 2024-09-22T03:22:15.633 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:25 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:15.722 INFO:teuthology.orchestra.run.smithi144.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 172 kB/s | 80 kB 00:00 2024-09-22T03:22:15.722 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:15.723 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.6 MB/s | 948 kB 00:00 2024-09-22T03:22:15.735 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:15.757 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:15.757 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:15.764 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:15.765 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:15.765 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-22T03:22:15.765 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:15.765 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:15.765 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-22T03:22:15.765 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:15.766 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:15.766 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:15.766 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:22:15.766 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:15.766 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 32 k 2024-09-22T03:22:15.766 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 142 k 2024-09-22T03:22:15.767 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:15.839 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:15.839 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:15.981 INFO:teuthology.orchestra.run.smithi129.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 148 kB/s | 32 kB 00:00 2024-09-22T03:22:15.981 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:15.982 INFO:teuthology.orchestra.run.smithi129.stdout:Total 147 kB/s | 32 kB 00:00 2024-09-22T03:22:15.982 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:15.986 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:15.986 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:16.000 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:16.001 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:16.028 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:16.087 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-22T03:22:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:22:16.569 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:16.803 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:16.803 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:16.803 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:16.803 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:16.803 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:16.803 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-22T03:22:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-22T03:22:16.986 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados2 2024-09-22T03:22:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-22T03:22:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:17.208 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-22T03:22:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:17.208 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:17.510 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-volume 2024-09-22T03:22:17.570 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:27 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:17.661 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:17.705 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:17.706 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:22:17.706 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:17.766 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librbd1 2024-09-22T03:22:18.101 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:26 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:18.240 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:18.240 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 285 k 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 285 k 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 1.3 M 2024-09-22T03:22:18.241 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:18.348 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:28 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:18.438 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:18.483 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:18.484 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:22:18.484 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:18.558 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rados 2024-09-22T03:22:18.752 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-volume-19.1.1-373.g4e45e6b4.el9.noarch.rpm 559 kB/s | 285 kB 00:00 2024-09-22T03:22:18.752 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:18.753 INFO:teuthology.orchestra.run.smithi144.stdout:Total 557 kB/s | 285 kB 00:00 2024-09-22T03:22:18.753 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:18.761 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:18.761 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:18.788 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:18.789 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:19.008 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:19.079 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:22:19.094 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:22:19.095 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:22:19.095 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-22T03:22:19.095 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:19.131 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:29 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:19.221 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:19.264 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:19.357 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:22:19.357 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:19.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rgw 2024-09-22T03:22:19.934 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:29 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:20.024 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:20.068 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:20.069 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:22:20.069 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:20.115 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-22T03:22:20.115 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:20.115 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:20.115 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-22T03:22:20.115 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:20.115 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:20.138 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-cephfs 2024-09-22T03:22:20.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados-devel 2024-09-22T03:22:20.716 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:30 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:20.805 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:20.848 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:20.849 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:22:20.849 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:20.913 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rbd 2024-09-22T03:22:21.010 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:29 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:21.146 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:22:21.147 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:21.148 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 127 k 2024-09-22T03:22:21.148 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 457 k 2024-09-22T03:22:21.148 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:21.365 INFO:teuthology.orchestra.run.smithi144.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 585 kB/s | 127 kB 00:00 2024-09-22T03:22:21.365 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:21.366 INFO:teuthology.orchestra.run.smithi144.stdout:Total 581 kB/s | 127 kB 00:00 2024-09-22T03:22:21.366 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:21.371 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:21.371 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:21.435 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:21.435 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:21.491 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:31 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:21.581 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:21.624 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:21.625 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-22T03:22:21.625 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:21.678 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:21.688 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-fuse 2024-09-22T03:22:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:22.276 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:32 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:22.406 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:22.407 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:22.407 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:22.407 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:22.407 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:22.407 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:22.419 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:22.419 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:22.419 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-22T03:22:22.419 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:22.420 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:22.420 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-22T03:22:22.420 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:22.420 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:22.420 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:22.420 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:22:22.420 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:22.421 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 85 k 2024-09-22T03:22:22.421 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 256 k 2024-09-22T03:22:22.421 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:22.593 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 496 kB/s | 85 kB 00:00 2024-09-22T03:22:22.594 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:22.594 INFO:teuthology.orchestra.run.smithi129.stdout:Total 490 kB/s | 85 kB 00:00 2024-09-22T03:22:22.595 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:22.601 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:22.602 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:22.640 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs2 2024-09-22T03:22:22.669 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:22.669 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:22.821 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:22.934 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:23.237 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:31 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:23.331 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:23.373 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:23.374 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:22:23.374 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:23.451 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs-devel 2024-09-22T03:22:23.594 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:23.594 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:23.594 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:23.594 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:23.594 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:23.594 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:23.828 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-mirror 2024-09-22T03:22:24.030 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:32 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:24.164 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:24.165 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:22:24.166 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:24.166 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 32 k 2024-09-22T03:22:24.166 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 142 k 2024-09-22T03:22:24.166 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:24.318 INFO:teuthology.orchestra.run.smithi144.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 208 kB/s | 32 kB 00:00 2024-09-22T03:22:24.319 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:24.319 INFO:teuthology.orchestra.run.smithi144.stdout:Total 206 kB/s | 32 kB 00:00 2024-09-22T03:22:24.319 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:24.323 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:24.323 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:24.336 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:24.336 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:24.424 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:24.428 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:34 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:24.533 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:24.568 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:24.569 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:24.570 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:22:24.570 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:24.570 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 3.1 M 2024-09-22T03:22:24.570 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 14 M 2024-09-22T03:22:24.570 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:24.888 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:24.897 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 9.3 MB/s | 3.1 MB 00:00 2024-09-22T03:22:24.898 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:24.898 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.3 MB/s | 3.1 MB 00:00 2024-09-22T03:22:24.898 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:24.907 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:24.907 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:24.988 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:24.989 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:25.123 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:25.123 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:25.123 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:25.123 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:25.123 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:25.123 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:25.325 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados2 2024-09-22T03:22:25.436 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:25.498 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:25.524 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:25.524 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:22:25.524 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-22T03:22:25.524 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:22:25.524 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:22:25.525 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:25.907 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:33 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:25.996 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:26.039 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:26.040 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:22:26.041 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:26.114 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librbd1 2024-09-22T03:22:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:26.653 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:26.653 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:26.654 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:26.699 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:34 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:26.790 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:26.835 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:26.836 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:22:26.836 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:26.884 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-nbd 2024-09-22T03:22:26.906 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rados 2024-09-22T03:22:27.481 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:35 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:27.485 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:02:37 ago on Sun 22 Sep 2024 03:19:50 AM UTC. 2024-09-22T03:22:27.570 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:27.614 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:27.615 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:22:27.615 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:27.625 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-22T03:22:27.626 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-22T03:22:27.626 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:27.626 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-22T03:22:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 169 k 2024-09-22T03:22:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:27.626 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-22T03:22:27.627 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-22T03:22:27.627 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-22T03:22:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:27.627 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 169 k 2024-09-22T03:22:27.627 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 540 k 2024-09-22T03:22:27.627 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-22T03:22:27.680 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rgw 2024-09-22T03:22:27.835 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 814 kB/s | 169 kB 00:00 2024-09-22T03:22:27.836 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:27.836 INFO:teuthology.orchestra.run.smithi129.stdout:Total 807 kB/s | 169 kB 00:00 2024-09-22T03:22:27.836 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-22T03:22:27.844 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-22T03:22:27.844 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-22T03:22:27.908 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-22T03:22:27.908 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-22T03:22:28.067 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-22T03:22:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:28.255 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:36 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:28.346 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:28.389 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:28.390 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:22:28.390 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:28.463 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-cephfs 2024-09-22T03:22:28.611 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:28.861 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:28.861 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:28.861 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-22T03:22:28.861 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:28.861 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:28.861 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-22T03:22:29.037 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:37 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:29.123 DEBUG:teuthology.parallel:result is None 2024-09-22T03:22:29.126 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:29.170 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:29.171 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:22:29.171 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:29.239 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rbd 2024-09-22T03:22:29.813 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:37 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:29.903 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-22T03:22:29.946 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:29.947 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-22T03:22:29.947 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:30.022 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-fuse 2024-09-22T03:22:30.611 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:38 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:30.749 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:30.749 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:30.749 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:22:30.750 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:30.751 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 85 k 2024-09-22T03:22:30.751 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 256 k 2024-09-22T03:22:30.751 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:30.947 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 437 kB/s | 85 kB 00:00 2024-09-22T03:22:30.947 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:30.948 INFO:teuthology.orchestra.run.smithi144.stdout:Total 433 kB/s | 85 kB 00:00 2024-09-22T03:22:30.949 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:30.954 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:30.955 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:31.019 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:31.020 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:31.171 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:31.681 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:31.945 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:31.945 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:31.945 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:31.945 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:31.946 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:31.946 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:32.205 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-mirror 2024-09-22T03:22:32.799 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:40 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:32.937 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:32.938 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:32.939 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:22:32.939 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:32.939 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 3.1 M 2024-09-22T03:22:32.939 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 14 M 2024-09-22T03:22:32.939 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:33.450 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 6.0 MB/s | 3.1 MB 00:00 2024-09-22T03:22:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:Total 6.0 MB/s | 3.1 MB 00:00 2024-09-22T03:22:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:33.460 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:33.460 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:33.540 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:33.541 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:33.983 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:34.043 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:34.068 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:34.068 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:22:34.069 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-22T03:22:34.069 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:22:34.069 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:22:34.069 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:35.184 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:35.185 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:35.454 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-nbd 2024-09-22T03:22:36.051 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:02:44 ago on Sun 22 Sep 2024 03:19:52 AM UTC. 2024-09-22T03:22:36.185 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-22T03:22:36.185 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-22T03:22:36.185 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:36.186 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-22T03:22:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 169 k 2024-09-22T03:22:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:36.186 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-22T03:22:36.186 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-22T03:22:36.186 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-22T03:22:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:36.187 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 169 k 2024-09-22T03:22:36.187 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 540 k 2024-09-22T03:22:36.187 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-22T03:22:36.396 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 810 kB/s | 169 kB 00:00 2024-09-22T03:22:36.396 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:22:36.397 INFO:teuthology.orchestra.run.smithi144.stdout:Total 804 kB/s | 169 kB 00:00 2024-09-22T03:22:36.397 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-22T03:22:36.404 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-22T03:22:36.405 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-22T03:22:36.469 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-22T03:22:36.469 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-22T03:22:36.620 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-22T03:22:36.727 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:37.191 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:37.462 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-22T03:22:37.462 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:37.462 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-22T03:22:37.462 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-22T03:22:37.462 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:37.462 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-22T03:22:37.723 DEBUG:teuthology.parallel:result is None 2024-09-22T03:22:37.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:22:37.871 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-22T03:22:37.897 INFO:teuthology.orchestra.run.smithi129.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-22T03:22:37.898 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-22T03:22:37.898 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-22T03:22:37.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T03:22:38.042 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-22T03:22:38.064 INFO:teuthology.orchestra.run.smithi144.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-22T03:22:38.065 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-22T03:22:38.065 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-22T03:22:38.066 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-22T03:22:38.067 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:38.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T03:22:38.097 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:38.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T03:22:38.131 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-22T03:22:38.132 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:38.132 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T03:22:38.166 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T03:22:38.234 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:38.234 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T03:22:38.260 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T03:22:38.324 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-22T03:22:38.324 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:38.324 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T03:22:38.351 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T03:22:38.417 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:38.417 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T03:22:38.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T03:22:38.511 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-22T03:22:38.511 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:38.511 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T03:22:38.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T03:22:38.603 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:38.603 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T03:22:38.628 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T03:22:38.692 INFO:teuthology.run_tasks:Running task ceph... 2024-09-22T03:22:38.775 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-22T03:22:38.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2024-09-22T03:22:38.778 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2024-09-22T03:22:38.801 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-22T03:22:38.802 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T03:22:38.843 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T03:22:38.867 INFO:tasks.ceph:Creating extra log directories... 2024-09-22T03:22:38.867 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T03:22:38.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T03:22:38.948 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-22T03:22:38.949 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-22T03:22:38.949 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757', 'branch': 'squid', 'description': 'crimson-rados/thrash_simple/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore thrashers/simple thrashosds-health workloads/radosbench-high-concurrency}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914757', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'squid', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1158, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'crimson-rados', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi129.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwS4KtJmnkd0Y7HAZ6MrpH+nNrcMjtptn2wen0MGqPAHFxjqhhIVKJvEGPxI2BSN1OQDjX7VHGhpUA9JCy7bPc=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHHZoZ6An1ukvNxbibSa2sboXGvQkbIV5wOVbVMlt0EXrZsXcZ1IPxKY5WrpjGYzmBlAf1cHutOqlMFyZSeAnpc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'full_sequential': [{'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}]}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:56:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-22T03:22:38.949 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T03:22:38.973 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T03:22:38.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T03:22:39.028 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T03:22:39.052 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:39.053 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T03:22:39.106 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T03:22:39.107 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-22T03:22:39.161 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T03:22:39.161 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:39.161 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-22T03:22:39.162 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:39.162 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:39.162 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-22 03:22:25.779185821 +0000 2024-09-22T03:22:39.162 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-22 03:19:11.693221550 +0000 2024-09-22T03:22:39.162 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-22 03:19:11.693221550 +0000 2024-09-22T03:22:39.162 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-22 03:19:11.693221550 +0000 2024-09-22T03:22:39.162 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T03:22:39.224 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-22T03:22:39.224 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-22T03:22:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000188021 s, 2.7 MB/s 2024-09-22T03:22:39.226 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T03:22:39.281 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-22 03:22:25.779185821 +0000 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-22 03:19:12.009214760 +0000 2024-09-22T03:22:39.337 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-22 03:19:12.009214760 +0000 2024-09-22T03:22:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-22 03:19:12.009214760 +0000 2024-09-22T03:22:39.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T03:22:39.400 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-22T03:22:39.400 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-22T03:22:39.400 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000185978 s, 2.8 MB/s 2024-09-22T03:22:39.402 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T03:22:39.458 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-22 03:22:25.779185821 +0000 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-22 03:19:12.328207906 +0000 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-22 03:19:12.328207906 +0000 2024-09-22T03:22:39.514 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-22 03:19:12.328207906 +0000 2024-09-22T03:22:39.515 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T03:22:39.577 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-22T03:22:39.577 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-22T03:22:39.577 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000187293 s, 2.7 MB/s 2024-09-22T03:22:39.578 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T03:22:39.634 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-22T03:22:39.689 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-22 03:22:25.780185799 +0000 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-22 03:19:12.650200987 +0000 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-22 03:19:12.650200987 +0000 2024-09-22T03:22:39.690 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-22 03:19:12.650200987 +0000 2024-09-22T03:22:39.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T03:22:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-22T03:22:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-22T03:22:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.00028151 s, 1.8 MB/s 2024-09-22T03:22:39.754 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T03:22:39.811 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:22:39.811 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:39.811 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T03:22:39.828 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T03:22:39.828 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-09-22T03:22:39.884 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-22 03:22:34.320003747 +0000 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-22 03:19:11.691397273 +0000 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-22 03:19:11.691397273 +0000 2024-09-22T03:22:39.885 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-22 03:19:11.691397273 +0000 2024-09-22T03:22:39.885 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T03:22:39.948 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-22T03:22:39.949 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-22T03:22:39.949 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000212699 s, 2.4 MB/s 2024-09-22T03:22:39.950 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T03:22:40.010 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-09-22T03:22:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T03:22:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:40.066 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-22T03:22:40.066 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:40.066 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:40.066 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-22 03:22:34.320003747 +0000 2024-09-22T03:22:40.066 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-22 03:19:12.010391775 +0000 2024-09-22T03:22:40.067 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-22 03:19:12.010391775 +0000 2024-09-22T03:22:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-22 03:19:12.010391775 +0000 2024-09-22T03:22:40.067 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T03:22:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-22T03:22:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-22T03:22:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000196991 s, 2.6 MB/s 2024-09-22T03:22:40.131 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T03:22:40.187 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-09-22T03:22:40.242 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T03:22:40.242 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:40.242 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-22T03:22:40.242 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:40.242 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:40.242 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-22 03:22:34.321003730 +0000 2024-09-22T03:22:40.243 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-22 03:19:12.331386242 +0000 2024-09-22T03:22:40.243 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-22 03:19:12.331386242 +0000 2024-09-22T03:22:40.243 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-22 03:19:12.331386242 +0000 2024-09-22T03:22:40.243 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T03:22:40.305 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-22T03:22:40.305 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-22T03:22:40.305 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000201354 s, 2.5 MB/s 2024-09-22T03:22:40.306 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T03:22:40.362 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-09-22T03:22:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T03:22:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:22:40.417 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-22T03:22:40.417 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:22:40.417 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:22:40.418 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-22 03:22:34.321003730 +0000 2024-09-22T03:22:40.418 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-22 03:19:12.640380915 +0000 2024-09-22T03:22:40.418 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-22 03:19:12.640380915 +0000 2024-09-22T03:22:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-22 03:19:12.640380915 +0000 2024-09-22T03:22:40.418 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T03:22:40.480 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-22T03:22:40.481 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-22T03:22:40.481 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000241185 s, 2.1 MB/s 2024-09-22T03:22:40.482 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T03:22:40.538 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:22:40.538 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-22T03:22:40.538 INFO:tasks.ceph:Generating config... 2024-09-22T03:22:40.539 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-22T03:22:40.539 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd delete sleep = 1 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd objectstore = seastore 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-22T03:22:40.540 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2024-09-22T03:22:40.541 INFO:tasks.ceph:[client.0] debug ms = 1 2024-09-22T03:22:40.541 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-09-22T03:22:40.541 INFO:tasks.ceph:[client.0] debug rados = 20 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] ms client mode = crc 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] ms service mode = crc 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2024-09-22T03:22:40.541 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2024-09-22T03:22:40.541 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-22T03:22:40.541 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-22T03:22:40.541 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-09-22T03:22:40.542 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-22T03:22:40.542 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-09-22T03:22:40.542 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-09-22T03:22:40.542 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-09-22T03:22:40.542 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-09-22T03:22:40.543 INFO:tasks.ceph:Setting up mon.a... 2024-09-22T03:22:40.543 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-22T03:22:40.634 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.keyring 2024-09-22T03:22:40.637 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-22T03:22:40.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:22:40.715 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.129')] 2024-09-22T03:22:40.716 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.129', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-09-22T03:22:40.717 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:40.717 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T03:22:40.771 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.129 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:40.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:40.855+0000 7fa3acc5fe80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:40.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:40.855+0000 7fa3acc5fe80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:40.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:40.855+0000 7fa3acc5fe80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: generated fsid 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:setting min_mon_release = quincy 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 0 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:fsid 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:last_changed 2024-09-22T03:22:40.857014+0000 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-09-22T03:22:40.857014+0000 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:min_mon_release 17 (quincy) 2024-09-22T03:22:40.858 INFO:teuthology.orchestra.run.smithi129.stdout:election_strategy: 1 2024-09-22T03:22:40.859 INFO:teuthology.orchestra.run.smithi129.stdout:0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-22T03:22:40.859 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-22T03:22:40.860 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T03:22:40.874 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7f5a2dae-b56a-4afe-bb9b-fb3f65651791... 2024-09-22T03:22:40.875 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T03:22:40.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout: chdir = "" 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout: auth supported = cephx 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout: filestore xattr use omap = true 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.957 INFO:teuthology.orchestra.run.smithi129.stdout: mon clock drift allowed = 1.000 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: osd crush chooseleaf type = 0 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: auth debug = true 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on old message = true 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on bug = true 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: mon pg warn max object skew = 0 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T03:22:40.958 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default size = 2 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow primary affinity = true 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow pg remap = true 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on legacy crush tunables = false 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on crush straw calc version zero = false 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on no sortbitwise = false 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on osd down out interval zero = false 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on too few osds = false 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: mon_allow_pool_size_one = true 2024-09-22T03:22:40.959 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: osd default data pool replay window = 5 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: mon allow pool delete = true 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: mon cluster log file level = debug 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: debug asserts on shutdown = true 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: mon health detail to clog = false 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: mon host = 172.21.15.129 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: ms client mode = crc 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: ms cluster mode = crc 2024-09-22T03:22:40.960 INFO:teuthology.orchestra.run.smithi129.stdout: ms mon client mode = crc 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: ms mon cluster mode = crc 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: ms mon service mode = crc 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: ms service mode = crc 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_min_size = 2 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_size = 3 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout:[osd] 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: osd journal size = 100 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub load threshold = 5.0 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub max interval = 120 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: osd mclock profile = high_recovery_ops 2024-09-22T03:22:40.961 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd recover clone overlap = true 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd recovery max chunk = 1048576 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug shutdown = true 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug op order = true 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug verify stray on activate = true 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug trim objects = true 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd open classes on start = true 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug pg log writeout = true 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd deep scrub update digest min age = 30 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.962 INFO:teuthology.orchestra.run.smithi129.stdout: osd map max advance = 10 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: journal zero on create = true 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: filestore ondisk finisher threads = 3 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: filestore apply finisher threads = 3 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: bdev debug aio = true 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug misdirected ops = true 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: debug monc = 20 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: crimson alien thread cpu cores = 6-7 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: crimson osd obc lru size = 10 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: debug osd = 20 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: osd blocked scrub grace period = 3600 2024-09-22T03:22:40.963 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug reject backfill probability = 0.3 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: osd delete sleep = 1 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: osd max backfills = 3 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: osd max markdown count = 1000 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: osd objectstore = seastore 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub min interval = 60 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: osd shutdown pgref assert = True 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: osd snap trim sleep = 2 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr] 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: debug mgr = 20 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:22:40.964 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: mgr/telemetry/nag = false 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout:[mon] 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: debug paxos = 20 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: mon data avail warn = 5 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: mon mgr mkfs grace = 240 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd reporter subtree level = osd 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd prime pg temp = true 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:22:40.965 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: # 1m isn't quite enough 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: mon_down_mkfs_grace = 2m 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_filestore_osds = false 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: mon min osdmap epochs = 50 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: mon osdmap full prune interval = 2 2024-09-22T03:22:40.966 INFO:teuthology.orchestra.run.smithi129.stdout: mon osdmap full prune min = 15 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: mon osdmap full prune txsize = 2 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default crimson = True 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: paxos service trim min = 10 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout:[client] 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: rgw cache enabled = true 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable ops log = true 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable usage log = true 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.a] 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-09-22T03:22:40.967 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout: debug objecter = 20 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout: debug rados = 20 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-09-22T03:22:40.968 INFO:teuthology.orchestra.run.smithi129.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-09-22T03:22:40.969 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-09-22T03:22:40.970 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-09-22T03:22:40.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = 172.21.15.129 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: ms client mode = crc 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: ms cluster mode = crc 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: ms mon client mode = crc 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: ms mon cluster mode = crc 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: ms mon service mode = crc 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: ms service mode = crc 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_min_size = 2 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_size = 3 2024-09-22T03:22:40.972 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 120 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-09-22T03:22:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-09-22T03:22:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: debug monc = 20 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: crimson alien thread cpu cores = 6-7 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: crimson osd obc lru size = 10 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd blocked scrub grace period = 3600 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug reject backfill probability = 0.3 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd delete sleep = 1 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd max backfills = 3 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd max markdown count = 1000 2024-09-22T03:22:40.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = seastore 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub min interval = 60 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: osd shutdown pgref assert = True 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: osd snap trim sleep = 2 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-09-22T03:22:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T03:22:40.977 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon min osdmap epochs = 50 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon osdmap full prune interval = 2 2024-09-22T03:22:40.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon osdmap full prune min = 15 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: mon osdmap full prune txsize = 2 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default crimson = True 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: paxos service trim min = 10 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: debug objecter = 20 2024-09-22T03:22:40.979 INFO:teuthology.orchestra.run.smithi144.stdout: debug rados = 20 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2024-09-22T03:22:40.980 INFO:teuthology.orchestra.run.smithi144.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:22:40.981 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-22T03:22:40.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-22T03:22:41.083 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-22T03:22:41.083 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:41.083 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-22T03:22:41.099 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:41.099 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-22T03:22:41.153 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2024-09-22T03:22:41.154 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:41.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T03:22:41.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:22:41.227 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:41.227 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:41.282 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2024-09-22T03:22:41.283 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:41.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T03:22:41.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:22:41.318 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:41.318 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:41.372 INFO:tasks.ceph:Setting up mon nodes... 2024-09-22T03:22:41.372 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-22T03:22:41.373 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T03:22:41.477 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T03:22:41.480 INFO:tasks.ceph:Setting up mds nodes... 2024-09-22T03:22:41.480 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-22T03:22:41.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-22T03:22:41.528 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-22T03:22:41.541 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-22T03:22:41.588 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-22T03:22:41.601 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-22T03:22:41.601 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-22T03:22:41.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-22T03:22:41.626 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:22:41.626 INFO:tasks.ceph:role: osd.0 2024-09-22T03:22:41.626 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-22T03:22:41.626 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T03:22:41.698 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:22:41.698 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:22:41.698 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:22:41.698 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:22:41.698 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:22:41.698 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:22:41.698 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:22:41.699 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:22:41.699 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:22:41.699 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:22:42.198 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-22T03:22:42.200 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-22T03:22:42.200 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-22T03:22:42.339 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-22T03:22:42.374 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-22T03:22:42.440 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:22:42.440 INFO:tasks.ceph:role: osd.1 2024-09-22T03:22:42.440 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-22T03:22:42.440 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T03:22:42.513 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:22:42.513 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:22:42.513 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:22:42.514 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:22:42.514 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:22:42.514 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:22:42.514 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:22:42.514 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:22:42.514 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:22:42.514 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:22:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-22T03:22:42.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-22T03:22:42.969 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-22T03:22:43.063 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-22T03:22:43.091 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:43.675 INFO:teuthology.orchestra.run.smithi129.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:43.676 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:43,676 seastar - Reactor backend: linux-aio 2024-09-22T03:22:43.705 INFO:teuthology.orchestra.run.smithi129.stderr:==27725==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:43.752 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:43,751 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:22:43.752 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:43,751 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-22T03:22:43.752 INFO:teuthology.orchestra.run.smithi129.stderr:DEBUG 2024-09-22 03:22:43,752 [shard 0:main] seastar - reactor::drain 2024-09-22T03:22:44.251 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:44.251 INFO:teuthology.orchestra.run.smithi129.stderr:Suppressions used: 2024-09-22T03:22:44.251 INFO:teuthology.orchestra.run.smithi129.stderr: count bytes template 2024-09-22T03:22:44.251 INFO:teuthology.orchestra.run.smithi129.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:44.251 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:44.251 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:22:44.305 INFO:teuthology.orchestra.run.smithi129.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:44.307 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:44,306 seastar - Reactor backend: linux-aio 2024-09-22T03:22:44.343 INFO:teuthology.orchestra.run.smithi129.stderr:==27721==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:44.493 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x94481 0x94d46 0x2ff768ac 0x2ff7730f 0x3014bdcc 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:44.494 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:44.494 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0xd5932 0xd5ac0 0xd554b 0x287b7 0xb738c 0x24eabcb7 0x24eabd21 0x3014abaf 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:44.494 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:44.799 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x161d27 0x94469 0x94d46 0x2ff35bd4 0x2ff36035 0x3014b5be 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd83d03 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:44.799 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0xb7ed9 0x28817 0xb738c 0x30145aed 0x3014b444 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:44.799 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:44.800 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:44.857 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c095afd 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047733a 0x2fdaa7b9 0x2fdac545 0x208bac63 0x2958f 0x2963f 0x20471d84 2024-09-22T03:22:44.857 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:45.529 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:45.529 INFO:teuthology.orchestra.run.smithi129.stderr:Suppressions used: 2024-09-22T03:22:45.529 INFO:teuthology.orchestra.run.smithi129.stderr: count bytes template 2024-09-22T03:22:45.529 INFO:teuthology.orchestra.run.smithi129.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:45.529 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:45.529 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:22:45.529 INFO:teuthology.orchestra.run.smithi129.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:45.572 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-22T03:22:45.597 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:46.141 INFO:teuthology.orchestra.run.smithi129.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:46.142 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:46,141 seastar - Reactor backend: linux-aio 2024-09-22T03:22:46.164 INFO:teuthology.orchestra.run.smithi129.stderr:==27774==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:46.208 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:46,208 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:22:46.208 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:46,208 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-22T03:22:46.209 INFO:teuthology.orchestra.run.smithi129.stderr:DEBUG 2024-09-22 03:22:46,208 [shard 0:main] seastar - reactor::drain 2024-09-22T03:22:46.692 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:46.692 INFO:teuthology.orchestra.run.smithi129.stderr:Suppressions used: 2024-09-22T03:22:46.692 INFO:teuthology.orchestra.run.smithi129.stderr: count bytes template 2024-09-22T03:22:46.692 INFO:teuthology.orchestra.run.smithi129.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:46.692 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:46.692 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:22:46.747 INFO:teuthology.orchestra.run.smithi129.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:46.749 INFO:teuthology.orchestra.run.smithi129.stderr:INFO 2024-09-22 03:22:46,748 seastar - Reactor backend: linux-aio 2024-09-22T03:22:46.776 INFO:teuthology.orchestra.run.smithi129.stderr:==27772==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:46.932 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x3004b0cf 0x3014aa5f 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:46.932 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:46.932 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2ff75e40 0x2ff7730f 0x3014bdcc 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:46.932 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:46.992 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2ffc2ed2 0x3014eb95 0x30153185 0x2ff3c411 0x2fd3eb74 0x2c46726d 0x2c471c9b 0x2baa3fbf 0x2b2893b7 0x2b3a7096 0x2b532008 0x30238775 0x302f6bae 0x304754e4 0x3047733a 0x2fdaa7b9 0x2fdac545 0x208bac63 0x2958f 0x2963f 0x20471d84 2024-09-22T03:22:46.993 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:47.250 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c559 0x2857c 0xb6323 0x30065068 0x300813a6 0x2b8ab682 0x2bd83c3f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:47.250 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:47.250 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x162b9f 0x2c6e3 0x2857c 0xb6323 0x2fcc5109 0x2fcc5a6e 0x2ffbd164 0x3014d7dd 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd83d03 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:47.250 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff8fc1933e 0xffffffff8fc20b38 0xffffffff8fb8eb59 0xffffffff8fb9699b 0xffffffff8fb96e1c 0xffffffff8f88c884 0xffffffff904d1db2 0xffffffff90600bb2 2024-09-22T03:22:47.308 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c0959a5 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047733a 0x2fdaa7b9 0x2fdac545 0x208bac63 0x2958f 0x2963f 0x20471d84 2024-09-22T03:22:47.308 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:47.340 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c095abe 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:47.340 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:47.340 INFO:teuthology.orchestra.run.smithi129.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c2d0569 0x2c095ceb 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:47.341 INFO:teuthology.orchestra.run.smithi129.stderr:kernel callstack: 2024-09-22T03:22:48.002 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:48.002 INFO:teuthology.orchestra.run.smithi129.stderr:Suppressions used: 2024-09-22T03:22:48.002 INFO:teuthology.orchestra.run.smithi129.stderr: count bytes template 2024-09-22T03:22:48.002 INFO:teuthology.orchestra.run.smithi129.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:48.002 INFO:teuthology.orchestra.run.smithi129.stderr:----------------------------------------------------- 2024-09-22T03:22:48.002 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:22:48.002 INFO:teuthology.orchestra.run.smithi129.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:48.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-22T03:22:48.071 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-22T03:22:48.101 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:22:48.101 INFO:tasks.ceph:role: osd.2 2024-09-22T03:22:48.101 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-22T03:22:48.101 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T03:22:48.178 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:22:48.178 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:22:48.178 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:22:48.178 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:22:48.178 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:22:48.179 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:22:48.179 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:22:48.179 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:22:48.179 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:22:48.179 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:22:48.689 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-22T03:22:48.692 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-22T03:22:48.692 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-22T03:22:48.846 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-22T03:22:48.881 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-22T03:22:48.950 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:22:48.950 INFO:tasks.ceph:role: osd.3 2024-09-22T03:22:48.950 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-22T03:22:48.950 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:22:49.025 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:22:49.468 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-22T03:22:49.471 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-22T03:22:49.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-09-22T03:22:49.568 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-22T03:22:49.599 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:50.180 INFO:teuthology.orchestra.run.smithi144.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:50.181 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:50,180 seastar - Reactor backend: linux-aio 2024-09-22T03:22:50.205 INFO:teuthology.orchestra.run.smithi144.stderr:==27357==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:50.251 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:50,250 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:22:50.251 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:50,250 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-22T03:22:50.251 INFO:teuthology.orchestra.run.smithi144.stderr:DEBUG 2024-09-22 03:22:50,251 [shard 0:main] seastar - reactor::drain 2024-09-22T03:22:50.740 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:50.740 INFO:teuthology.orchestra.run.smithi144.stderr:Suppressions used: 2024-09-22T03:22:50.740 INFO:teuthology.orchestra.run.smithi144.stderr: count bytes template 2024-09-22T03:22:50.740 INFO:teuthology.orchestra.run.smithi144.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:50.740 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:50.741 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-22T03:22:50.790 INFO:teuthology.orchestra.run.smithi144.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:50.791 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:50,791 seastar - Reactor backend: linux-aio 2024-09-22T03:22:50.819 INFO:teuthology.orchestra.run.smithi144.stderr:==27355==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:50.975 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2ff763e7 0x2ff7730f 0x2ff775b5 0x3014ba39 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:50.976 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:50.976 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2b0e4609 0x2b8ab21f 0x2bd807cb 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:50.976 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:51.278 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x301534d6 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:51.278 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x3973f 0x94456 0x94d46 0x2ff34d86 0x30080c52 0x2b8ab682 0x2bd7eef6 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:51.278 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:51.278 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:51.336 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c095b0a 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047733a 0x2fdaa7b9 0x2fdac545 0x208bac63 0x2958f 0x2963f 0x20471d84 2024-09-22T03:22:51.336 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:52.005 INFO:teuthology.orchestra.run.smithi144.stderr:Suppressions used: 2024-09-22T03:22:52.005 INFO:teuthology.orchestra.run.smithi144.stderr: count bytes template 2024-09-22T03:22:52.005 INFO:teuthology.orchestra.run.smithi144.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:52.005 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:52.005 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-22T03:22:52.005 INFO:teuthology.orchestra.run.smithi144.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:52.050 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-22T03:22:52.075 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:52.622 INFO:teuthology.orchestra.run.smithi144.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:52.623 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:52,622 seastar - Reactor backend: linux-aio 2024-09-22T03:22:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:==27405==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:52.689 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:52,688 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:22:52.689 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:52,688 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-22T03:22:52.689 INFO:teuthology.orchestra.run.smithi144.stderr:DEBUG 2024-09-22 03:22:52,688 [shard 0:main] seastar - reactor::drain 2024-09-22T03:22:53.177 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:53.178 INFO:teuthology.orchestra.run.smithi144.stderr:Suppressions used: 2024-09-22T03:22:53.178 INFO:teuthology.orchestra.run.smithi144.stderr: count bytes template 2024-09-22T03:22:53.178 INFO:teuthology.orchestra.run.smithi144.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:53.178 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:53.178 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-22T03:22:53.233 INFO:teuthology.orchestra.run.smithi144.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:22:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:INFO 2024-09-22 03:22:53,234 seastar - Reactor backend: linux-aio 2024-09-22T03:22:53.262 INFO:teuthology.orchestra.run.smithi144.stderr:==27403==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:22:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2ff76204 0x2ffb4056 0x2ffbdf40 0x3014d7dd 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x203da87f 0x2ff75c96 0x3014c053 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd80a7f 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2acacc1c 0x2b242299 0x2b3b1886 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:53.478 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c605 0x2857c 0xb6323 0x2c5055fd 0x2c317d01 0x2c14ca30 0x2c46f996 0x2baa3fbf 0x2b2893b7 0x2b3a7096 0x2b532008 0x30238775 0x302f6bae 0x304754e4 0x3047733a 0x2fdaa7b9 0x2fdac545 0x208bac63 0x2958f 0x2963f 0x20471d84 2024-09-22T03:22:53.478 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff869e3f7a 0xffffffff867cf7db 0xffffffff8678e0cf 0xffffffff8679699b 0xffffffff86796e1c 0xffffffff8648c884 0xffffffff870d1db2 0xffffffff87200bb2 2024-09-22T03:22:53.740 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2ff751e3 0x2ff7730f 0x2ff775b5 0x3014ba39 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd83d03 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:53.740 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2ffb0ef2 0x2ffc4e07 0x3014e2a4 0x30153185 0x2ff3c411 0x2fd3eb74 0x2bd83d03 0x2bd97077 0x2bb2a603 0x2b2859fc 0x2aca5660 0x2b24a027 0x2b39eea2 0x2b60ab92 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:53.740 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:53.740 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:53.797 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c095a9f 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047733a 0x2fdaa7b9 0x2fdac545 0x208bac63 0x2958f 0x2963f 0x20471d84 2024-09-22T03:22:53.797 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:53.829 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c095ac4 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:53.830 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:53.830 INFO:teuthology.orchestra.run.smithi144.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: 0x45d5d 0x302dc944 0x300f44c5 0x3021b180 0x3021b504 0x3021b64e 0x3021baa4 0x3e6ef 0x2c095a19 0x2c21e1fb 0x2c132403 0x2ba0ddbf 0x2ba105d4 0x2ba10f8b 0x2ba11587 0x2bb947dc 0x2b50cf98 0x30238775 0x302f6bae 0x304754e4 0x3047b292 0x3047b814 0x30fab151 0x89c51 0x10ec7f 2024-09-22T03:22:53.830 INFO:teuthology.orchestra.run.smithi144.stderr:kernel callstack: 2024-09-22T03:22:54.494 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:54.495 INFO:teuthology.orchestra.run.smithi144.stderr:Suppressions used: 2024-09-22T03:22:54.495 INFO:teuthology.orchestra.run.smithi144.stderr: count bytes template 2024-09-22T03:22:54.495 INFO:teuthology.orchestra.run.smithi144.stderr: 1 8 MallocExtension::Register 2024-09-22T03:22:54.495 INFO:teuthology.orchestra.run.smithi144.stderr:----------------------------------------------------- 2024-09-22T03:22:54.495 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-22T03:22:54.495 INFO:teuthology.orchestra.run.smithi144.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:22:54.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-22T03:22:54.564 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-22T03:22:54.564 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:54.564 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-22T03:22:54.590 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:54.590 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-22T03:22:54.655 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:54.656 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-22T03:22:54.680 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:54.680 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-22T03:22:54.744 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:54.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-22T03:22:54.808 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:54.808 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-22T03:22:54.823 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:54.823 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-22T03:22:54.863 INFO:tasks.ceph:Adding keys to all mons... 2024-09-22T03:22:54.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCEje9mRAPAFxAAZjFGkkIjnEvgHunWq/Fx4Q== 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCGje9mGHahMRAA5Y7FDc16RctBiJtVhXB3AA== 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCBje9mkzZqHBAAQOew6i9Y550rSgMuTqlhXQ== 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCKje9mdJ8xNBAAMDaSEOnd9Vf/tBLh9fLo/g== 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCNje9mVyIFExAAuV5o55juybD6pVUfhPkzsQ== 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-09-22T03:22:54.887 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCBje9mb3B4HxAAMVb0J4XS5CBxYFD6zkhovA== 2024-09-22T03:22:54.888 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2024-09-22T03:22:54.888 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQCBje9mawQPIxAAIy4ewqQYebgnmOSq9cCxBA== 2024-09-22T03:22:54.888 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:22:54.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:22:55.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T03:22:55.097 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:22:55.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:22:55.210 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T03:22:55.269 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T03:22:55.329 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-22T03:22:55.329 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-22T03:22:55.354 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T03:22:55.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:55.453+0000 7fdeb088ab40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:55.453+0000 7fdeb088ab40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.468 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-22T03:22:55.492 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:22:55.547 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-22T03:22:55.548 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-22T03:22:55.548 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-22T03:22:55.590 INFO:tasks.ceph.mon.a:Started 2024-09-22T03:22:55.590 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-22T03:22:55.590 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-22T03:22:55.590 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-22T03:22:55.594 INFO:tasks.ceph.mgr.x:Started 2024-09-22T03:22:55.594 INFO:tasks.ceph:Setting crush tunables to default 2024-09-22T03:22:55.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd crush tunables default 2024-09-22T03:22:55.666 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-09-22T03:22:55.663+0000 7f9a2ac8ab40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.670 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-09-22T03:22:55.663+0000 7f9a2ac8ab40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.671 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-09-22T03:22:55.664+0000 7f9a2ac8ab40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.683 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:55.681+0000 7fd1861c1f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:55.692+0000 7fb790b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.710 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:55.709+0000 7fd1861c1f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:55.709+0000 7fb790b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.711 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:55.709+0000 7fd1861c1f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.777 INFO:teuthology.orchestra.run.smithi129.stderr:adjusted tunables profile to default 2024-09-22T03:22:55.788 INFO:tasks.ceph:check_enable_crimson: True 2024-09-22T03:22:55.789 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-09-22T03:22:55.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-09-22T03:22:55.810 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:55.809+0000 7fd1861c1f00 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-22T03:22:55.851 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:55.849+0000 7fd1861c1f00 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-22T03:22:55.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:55.884+0000 7feb84997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:55.892+0000 7feb84997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:55.969 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-09-22T03:22:55.968+0000 7f9a21ff1640 -1 WARNING: experimental feature 'crimson' is enabled 2024-09-22T03:22:55.969 INFO:tasks.ceph.mon.a.smithi129.stderr:Please be aware that this feature is experimental, untested, 2024-09-22T03:22:55.969 INFO:tasks.ceph.mon.a.smithi129.stderr:unsupported, and may result in data corruption, data loss, 2024-09-22T03:22:55.969 INFO:tasks.ceph.mon.a.smithi129.stderr:and/or irreparable damage to your cluster. Do not use 2024-09-22T03:22:55.970 INFO:tasks.ceph.mon.a.smithi129.stderr:feature with important data. 2024-09-22T03:22:55.970 INFO:tasks.ceph.mon.a.smithi129.stderr: 2024-09-22T03:22:55.986 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:55.985+0000 7fd1861c1f00 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-22T03:22:56.036 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:56.034+0000 7fd1861c1f00 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-22T03:22:56.193 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:56.191+0000 7fd1861c1f00 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-22T03:22:56.302 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:56.300+0000 7fd1861c1f00 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-22T03:22:56.630 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:56.628+0000 7fd1861c1f00 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-22T03:22:56.684 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:56.683+0000 7fd1861c1f00 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-22T03:22:56.792 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-22T03:22:56.792 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:56.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-22T03:22:56.820 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-22T03:22:56.821 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-22T03:22:56.832 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:56.831+0000 7fd1861c1f00 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-22T03:22:56.887 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:56.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-22T03:22:56.913 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-22T03:22:56.913 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-22T03:22:56.979 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new b141605f-6391-41b8-b680-cb81bf703b22 0 2024-09-22T03:22:57.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.116+0000 7f8f3a8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.125+0000 7f8f3a8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.213 INFO:teuthology.orchestra.run.smithi144.stdout:0 2024-09-22T03:22:57.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new a1a59185-9c42-4359-af1f-3dd8be188aa7 1 2024-09-22T03:22:57.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.322+0000 7f06e4e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.331+0000 7f06e4e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.406 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-09-22T03:22:57.418 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new c1b83060-4307-4966-a545-5476767a17c1 2 2024-09-22T03:22:57.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.515+0000 7fafe7223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.523+0000 7fafe7223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.601 INFO:teuthology.orchestra.run.smithi144.stdout:2 2024-09-22T03:22:57.613 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 487e7a88-d3c1-469d-83b4-222cb8676d02 3 2024-09-22T03:22:57.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.710+0000 7f7898ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-22T03:22:57.718+0000 7f7898ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:57.796 INFO:teuthology.orchestra.run.smithi144.stdout:3 2024-09-22T03:22:57.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-09-22T03:22:57.831 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:57.830+0000 7fd1861c1f00 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-22T03:22:57.841 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:22:57.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:22:57.884 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:57.882+0000 7fd1861c1f00 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-22T03:22:57.884 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:22:57.884 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-09-22T03:22:57.916 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:22:57.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:22:57.958 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:22:57.958 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-09-22T03:22:57.988 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:22:57.988 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:22:58.030 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:22:58.031 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-09-22T03:22:58.059 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:22:58.059 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:22:58.102 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:22:58.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:22:58.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:58.217+0000 7f8d44d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:22:58.226+0000 7f8d44d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:58.292 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.290+0000 7fd1861c1f00 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-22T03:22:58.307 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:22:58.307 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:22:57.794142+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:22:58.321 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-22T03:22:58.321 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-22T03:22:58.395 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.393+0000 7fd1861c1f00 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-22T03:22:58.517 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.516+0000 7fd1861c1f00 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-22T03:22:58.569 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.568+0000 7fd1861c1f00 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-22T03:22:58.694 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.693+0000 7fd1861c1f00 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-22T03:22:58.724 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T03:22:58.751 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.749+0000 7fd1861c1f00 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-22T03:22:58.803 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.799+0000 7fd1861c1f00 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-22T03:22:58.825 INFO:teuthology.misc.health.smithi129.stderr:2024-09-22T03:22:58.819+0000 7ff5c006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:58.828 INFO:teuthology.misc.health.smithi129.stderr:2024-09-22T03:22:58.826+0000 7ff5c006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:22:58.893 INFO:teuthology.misc.health.smithi129.stdout: 2024-09-22T03:22:58.893 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":7,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:22:57.794142+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:22:58.903 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-22T03:22:58.922 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.921+0000 7fd1861c1f00 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-22T03:22:58.977 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:58.976+0000 7fd1861c1f00 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-22T03:22:59.033 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:59.032+0000 7fd1861c1f00 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-22T03:22:59.107 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:59.106+0000 7fd1861c1f00 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-22T03:22:59.156 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:59.155+0000 7fd1861c1f00 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-22T03:22:59.204 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:59.203+0000 7fd1861c1f00 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-22T03:22:59.927 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:22:59.926+0000 7fd1861c1f00 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-22T03:23:00.939 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-22T03:23:00.938+0000 7fd124a19640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-22T03:23:05.306 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T03:23:05.403 INFO:teuthology.misc.health.smithi129.stderr:2024-09-22T03:23:05.398+0000 7f5ed4f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:05.405 INFO:teuthology.misc.health.smithi129.stderr:2024-09-22T03:23:05.404+0000 7f5ed4f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:05.582 INFO:teuthology.misc.health.smithi129.stdout: 2024-09-22T03:23:05.583 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":12,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:23:04.800320+0000","last_up_change":"2024-09-22T03:23:01.787205+0000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:23:01.973068+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":27947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":27947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":27947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":27947901}]},"public_addr":"172.21.15.129:6800/27947901","cluster_addr":"172.21.15.129:6801/27947901","heartbeat_back_addr":"172.21.15.129:6805/27947901","heartbeat_front_addr":"172.21.15.129:6802/27947901","state":["exists","up"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2887884686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2887884686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2887884686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2887884686}]},"public_addr":"172.21.15.129:6803/2887884686","cluster_addr":"172.21.15.129:6804/2887884686","heartbeat_back_addr":"172.21.15.129:6806/2887884686","heartbeat_front_addr":"172.21.15.129:6807/2887884686","state":["exists","up"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3028982311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":3028982311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3028982311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3028982311}]},"public_addr":"172.21.15.144:6804/3028982311","cluster_addr":"172.21.15.144:6803/3028982311","heartbeat_back_addr":"172.21.15.144:6814/3028982311","heartbeat_front_addr":"172.21.15.144:6815/3028982311","state":["exists","up"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1383918015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":1383918015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1383918015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":1383918015}]},"public_addr":"172.21.15.144:6806/1383918015","cluster_addr":"172.21.15.144:6807/1383918015","heartbeat_back_addr":"172.21.15.144:6808/1383918015","heartbeat_front_addr":"172.21.15.144:6805/1383918015","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:23:05.594 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-22T03:23:05.594 INFO:tasks.ceph:Creating RBD pool 2024-09-22T03:23:05.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-22T03:23:05.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:05.689+0000 7ff301a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:05.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:05.699+0000 7ff301a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:06.809 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'rbd' created 2024-09-22T03:23:06.829 DEBUG:teuthology.orchestra.run.smithi129:> rbd --cluster ceph pool init rbd 2024-09-22T03:23:06.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:06.892+0000 7f375ef61d00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:06.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:06.892+0000 7f375ef61d00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:06.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:06.896+0000 7f375ef61d00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:09.862 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-22T03:23:09.863 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-22T03:23:09.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-22T03:23:09.863 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-22T03:23:09.863 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-22T03:23:09.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:09.965+0000 7fee8c64f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:09.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:09.972+0000 7fee8c64f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.195 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:10.207 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1128335265},{"type":"v1","addr":"172.21.15.144:6801","nonce":1128335265}]},"active_addr":"172.21.15.144:6801/1128335265","active_change":"2024-09-22T03:22:59.928958+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":362677745}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":3144477838}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":20622194}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":1677681010}]}]} 2024-09-22T03:23:10.209 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-22T03:23:10.209 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T03:23:10.209 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:23:10.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.307+0000 7f73f282c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.314+0000 7f73f282c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.491 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:10.492 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":16,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:23:09.819356+0000","last_up_change":"2024-09-22T03:23:01.787205+0000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:23:01.973068+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:23:05.895212+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":27947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":27947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":27947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":27947901}]},"public_addr":"172.21.15.129:6800/27947901","cluster_addr":"172.21.15.129:6801/27947901","heartbeat_back_addr":"172.21.15.129:6805/27947901","heartbeat_front_addr":"172.21.15.129:6802/27947901","state":["exists","up"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2887884686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2887884686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2887884686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2887884686}]},"public_addr":"172.21.15.129:6803/2887884686","cluster_addr":"172.21.15.129:6804/2887884686","heartbeat_back_addr":"172.21.15.129:6806/2887884686","heartbeat_front_addr":"172.21.15.129:6807/2887884686","state":["exists","up"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3028982311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":3028982311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3028982311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3028982311}]},"public_addr":"172.21.15.144:6804/3028982311","cluster_addr":"172.21.15.144:6803/3028982311","heartbeat_back_addr":"172.21.15.144:6814/3028982311","heartbeat_front_addr":"172.21.15.144:6815/3028982311","state":["exists","up"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1383918015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":1383918015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1383918015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":1383918015}]},"public_addr":"172.21.15.144:6806/1383918015","cluster_addr":"172.21.15.144:6807/1383918015","heartbeat_back_addr":"172.21.15.144:6808/1383918015","heartbeat_front_addr":"172.21.15.144:6805/1383918015","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:23:10.504 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T03:23:10.505 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:23:10.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.602+0000 7f7e1bf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.608+0000 7f7e1bf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.787 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:10.787 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":16,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:23:09.819356+0000","last_up_change":"2024-09-22T03:23:01.787205+0000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:23:01.973068+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:23:05.895212+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":27947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":27947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":27947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":27947901}]},"public_addr":"172.21.15.129:6800/27947901","cluster_addr":"172.21.15.129:6801/27947901","heartbeat_back_addr":"172.21.15.129:6805/27947901","heartbeat_front_addr":"172.21.15.129:6802/27947901","state":["exists","up"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2887884686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2887884686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2887884686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2887884686}]},"public_addr":"172.21.15.129:6803/2887884686","cluster_addr":"172.21.15.129:6804/2887884686","heartbeat_back_addr":"172.21.15.129:6806/2887884686","heartbeat_front_addr":"172.21.15.129:6807/2887884686","state":["exists","up"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3028982311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":3028982311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3028982311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3028982311}]},"public_addr":"172.21.15.144:6804/3028982311","cluster_addr":"172.21.15.144:6803/3028982311","heartbeat_back_addr":"172.21.15.144:6814/3028982311","heartbeat_front_addr":"172.21.15.144:6815/3028982311","state":["exists","up"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1383918015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":1383918015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1383918015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":1383918015}]},"public_addr":"172.21.15.144:6806/1383918015","cluster_addr":"172.21.15.144:6807/1383918015","heartbeat_back_addr":"172.21.15.144:6808/1383918015","heartbeat_front_addr":"172.21.15.144:6805/1383918015","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:23:10.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T03:23:10.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T03:23:10.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T03:23:10.801 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T03:23:10.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.911+0000 7f9c88e58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.911+0000 7fe318d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.911+0000 7f89d8157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.911+0000 7f0533517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.919+0000 7f9c88e58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.920+0000 7f0533517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.920+0000 7fe318d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:10.920+0000 7f89d8157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:10.952 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:10.952 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:23:10.953 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:10.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:23:10.955 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:10.955 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:23:10.957 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:10.957 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:23:11.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.055+0000 7fb70ff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.055+0000 7fdeb96a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.064+0000 7fb70ff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.065+0000 7fdeb96a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.103+0000 7f7cc4b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.103+0000 7f677b26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.111+0000 7f677b26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.111+0000 7f7cc4b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.261 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:11.270 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:11.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705673 for osd.1 2024-09-22T03:23:11.275 DEBUG:teuthology.parallel:result is None 2024-09-22T03:23:11.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705673 for osd.3 2024-09-22T03:23:11.283 DEBUG:teuthology.parallel:result is None 2024-09-22T03:23:11.299 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:38654705673 2024-09-22T03:23:11.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705673 for osd.0 2024-09-22T03:23:11.313 DEBUG:teuthology.parallel:result is None 2024-09-22T03:23:11.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705673 for osd.2 2024-09-22T03:23:11.317 DEBUG:teuthology.parallel:result is None 2024-09-22T03:23:11.317 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T03:23:11.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:11.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.455+0000 7f0d59058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.461+0000 7f0d59058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:11.637 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:11.649 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-22T03:23:10.942594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":5112,"kb_used_data":5112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209185800,"statfs":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841697+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:09.841697+0000","last_peered":"2024-09-22T03:23:09.841697+0000","last_clean":"2024-09-22T03:23:09.841697+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:09.841697+0000","last_undegraded":"2024-09-22T03:23:09.841697+0000","last_fullsized":"2024-09-22T03:23:09.841697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837288+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:09.837288+0000","last_peered":"2024-09-22T03:23:09.837288+0000","last_clean":"2024-09-22T03:23:09.837288+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:09.837288+0000","last_undegraded":"2024-09-22T03:23:09.837288+0000","last_fullsized":"2024-09-22T03:23:09.837288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.836988+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.836988+0000","last_peered":"2024-09-22T03:23:09.836988+0000","last_clean":"2024-09-22T03:23:09.836988+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:09.836988+0000","last_undegraded":"2024-09-22T03:23:09.836988+0000","last_fullsized":"2024-09-22T03:23:09.836988+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.837266+0000","last_peered":"2024-09-22T03:23:09.837266+0000","last_clean":"2024-09-22T03:23:09.837266+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:09.837266+0000","last_undegraded":"2024-09-22T03:23:09.837266+0000","last_fullsized":"2024-09-22T03:23:09.837266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'1","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841202+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.841202+0000","last_peered":"2024-09-22T03:23:09.841202+0000","last_clean":"2024-09-22T03:23:09.841202+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:09.841202+0000","last_undegraded":"2024-09-22T03:23:09.841202+0000","last_fullsized":"2024-09-22T03:23:09.841202+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.838867+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.838867+0000","last_peered":"2024-09-22T03:23:09.838867+0000","last_clean":"2024-09-22T03:23:09.838867+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:09.838867+0000","last_undegraded":"2024-09-22T03:23:09.838867+0000","last_fullsized":"2024-09-22T03:23:09.838867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842602+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:09.842602+0000","last_peered":"2024-09-22T03:23:09.842602+0000","last_clean":"2024-09-22T03:23:09.842602+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:09.842602+0000","last_undegraded":"2024-09-22T03:23:09.842602+0000","last_fullsized":"2024-09-22T03:23:09.842602+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842517+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:09.842517+0000","last_peered":"2024-09-22T03:23:09.842517+0000","last_clean":"2024-09-22T03:23:09.842517+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:09.842517+0000","last_undegraded":"2024-09-22T03:23:09.842517+0000","last_fullsized":"2024-09-22T03:23:09.842517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":151,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.835345+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:09.835345+0000","last_peered":"2024-09-22T03:23:09.835345+0000","last_clean":"2024-09-22T03:23:09.835345+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:09.835345+0000","last_undegraded":"2024-09-22T03:23:09.835345+0000","last_fullsized":"2024-09-22T03:23:09.835345+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1244,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296484,"statfs":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1632,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296096,"statfs":{"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":668,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297060,"statfs":{"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1568,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296160,"statfs":{"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:11.650 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:11.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.748+0000 7fc14dd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:11.755+0000 7fc14dd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:11.932 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:11.944 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-22T03:23:10.942594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":5112,"kb_used_data":5112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209185800,"statfs":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841697+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:09.841697+0000","last_peered":"2024-09-22T03:23:09.841697+0000","last_clean":"2024-09-22T03:23:09.841697+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:09.841697+0000","last_undegraded":"2024-09-22T03:23:09.841697+0000","last_fullsized":"2024-09-22T03:23:09.841697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837288+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:09.837288+0000","last_peered":"2024-09-22T03:23:09.837288+0000","last_clean":"2024-09-22T03:23:09.837288+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:09.837288+0000","last_undegraded":"2024-09-22T03:23:09.837288+0000","last_fullsized":"2024-09-22T03:23:09.837288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.836988+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.836988+0000","last_peered":"2024-09-22T03:23:09.836988+0000","last_clean":"2024-09-22T03:23:09.836988+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:09.836988+0000","last_undegraded":"2024-09-22T03:23:09.836988+0000","last_fullsized":"2024-09-22T03:23:09.836988+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.837266+0000","last_peered":"2024-09-22T03:23:09.837266+0000","last_clean":"2024-09-22T03:23:09.837266+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:09.837266+0000","last_undegraded":"2024-09-22T03:23:09.837266+0000","last_fullsized":"2024-09-22T03:23:09.837266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'1","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841202+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.841202+0000","last_peered":"2024-09-22T03:23:09.841202+0000","last_clean":"2024-09-22T03:23:09.841202+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:09.841202+0000","last_undegraded":"2024-09-22T03:23:09.841202+0000","last_fullsized":"2024-09-22T03:23:09.841202+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.838867+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.838867+0000","last_peered":"2024-09-22T03:23:09.838867+0000","last_clean":"2024-09-22T03:23:09.838867+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:09.838867+0000","last_undegraded":"2024-09-22T03:23:09.838867+0000","last_fullsized":"2024-09-22T03:23:09.838867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842602+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:09.842602+0000","last_peered":"2024-09-22T03:23:09.842602+0000","last_clean":"2024-09-22T03:23:09.842602+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:09.842602+0000","last_undegraded":"2024-09-22T03:23:09.842602+0000","last_fullsized":"2024-09-22T03:23:09.842602+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842517+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:09.842517+0000","last_peered":"2024-09-22T03:23:09.842517+0000","last_clean":"2024-09-22T03:23:09.842517+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:09.842517+0000","last_undegraded":"2024-09-22T03:23:09.842517+0000","last_fullsized":"2024-09-22T03:23:09.842517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":151,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.835345+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:09.835345+0000","last_peered":"2024-09-22T03:23:09.835345+0000","last_clean":"2024-09-22T03:23:09.835345+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:09.835345+0000","last_undegraded":"2024-09-22T03:23:09.835345+0000","last_fullsized":"2024-09-22T03:23:09.835345+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1244,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296484,"statfs":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1632,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296096,"statfs":{"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":668,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297060,"statfs":{"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1568,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296160,"statfs":{"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:11.945 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T03:23:11.945 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-22T03:23:11.945 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-22T03:23:11.945 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-22T03:23:12.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:12.044+0000 7fca0c058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:12.050+0000 7fca0c058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:12.261 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:12.262 INFO:teuthology.orchestra.run.smithi129.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-22T03:23:12.274 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-22T03:23:12.274 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-22T03:23:12.351 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-22T03:23:12.351 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'squid', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T03:23:12.351 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_down': 0, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-22T03:23:12.351 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-22T03:23:12.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:12.450+0000 7fe2f6f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:12.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:12.460+0000 7fe2f6f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:12.655 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-22T03:23:12.655 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T03:23:12.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:12.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:12.754+0000 7f6c184a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:12.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:12.760+0000 7f6c184a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:12.938 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:12.938 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:12.951 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-22T03:23:11.937433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":5112,"kb_used_data":5112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209185800,"statfs":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994839"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841697+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:09.841697+0000","last_peered":"2024-09-22T03:23:09.841697+0000","last_clean":"2024-09-22T03:23:09.841697+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:09.841697+0000","last_undegraded":"2024-09-22T03:23:09.841697+0000","last_fullsized":"2024-09-22T03:23:09.841697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837288+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:09.837288+0000","last_peered":"2024-09-22T03:23:09.837288+0000","last_clean":"2024-09-22T03:23:09.837288+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:09.837288+0000","last_undegraded":"2024-09-22T03:23:09.837288+0000","last_fullsized":"2024-09-22T03:23:09.837288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.836988+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.836988+0000","last_peered":"2024-09-22T03:23:09.836988+0000","last_clean":"2024-09-22T03:23:09.836988+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:09.836988+0000","last_undegraded":"2024-09-22T03:23:09.836988+0000","last_fullsized":"2024-09-22T03:23:09.836988+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.837266+0000","last_peered":"2024-09-22T03:23:09.837266+0000","last_clean":"2024-09-22T03:23:09.837266+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:09.837266+0000","last_undegraded":"2024-09-22T03:23:09.837266+0000","last_fullsized":"2024-09-22T03:23:09.837266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'1","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841202+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.841202+0000","last_peered":"2024-09-22T03:23:09.841202+0000","last_clean":"2024-09-22T03:23:09.841202+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:09.841202+0000","last_undegraded":"2024-09-22T03:23:09.841202+0000","last_fullsized":"2024-09-22T03:23:09.841202+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.838867+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.838867+0000","last_peered":"2024-09-22T03:23:09.838867+0000","last_clean":"2024-09-22T03:23:09.838867+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:09.838867+0000","last_undegraded":"2024-09-22T03:23:09.838867+0000","last_fullsized":"2024-09-22T03:23:09.838867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842602+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:09.842602+0000","last_peered":"2024-09-22T03:23:09.842602+0000","last_clean":"2024-09-22T03:23:09.842602+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:09.842602+0000","last_undegraded":"2024-09-22T03:23:09.842602+0000","last_fullsized":"2024-09-22T03:23:09.842602+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842517+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:09.842517+0000","last_peered":"2024-09-22T03:23:09.842517+0000","last_clean":"2024-09-22T03:23:09.842517+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:09.842517+0000","last_undegraded":"2024-09-22T03:23:09.842517+0000","last_fullsized":"2024-09-22T03:23:09.842517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":151,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.835345+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:09.835345+0000","last_peered":"2024-09-22T03:23:09.835345+0000","last_clean":"2024-09-22T03:23:09.835345+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:09.835345+0000","last_undegraded":"2024-09-22T03:23:09.835345+0000","last_fullsized":"2024-09-22T03:23:09.835345+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1244,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296484,"statfs":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1632,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296096,"statfs":{"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":668,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297060,"statfs":{"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1568,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296160,"statfs":{"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:12.952 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:13.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:13.049+0000 7f761e658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:13.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:13.056+0000 7f761e658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:13.232 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:13.244 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-22T03:23:11.937433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":5112,"kb_used_data":5112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209185800,"statfs":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994839"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841697+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:09.841697+0000","last_peered":"2024-09-22T03:23:09.841697+0000","last_clean":"2024-09-22T03:23:09.841697+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:09.841697+0000","last_undegraded":"2024-09-22T03:23:09.841697+0000","last_fullsized":"2024-09-22T03:23:09.841697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837288+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:09.837288+0000","last_peered":"2024-09-22T03:23:09.837288+0000","last_clean":"2024-09-22T03:23:09.837288+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:09.837288+0000","last_undegraded":"2024-09-22T03:23:09.837288+0000","last_fullsized":"2024-09-22T03:23:09.837288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.836988+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.836988+0000","last_peered":"2024-09-22T03:23:09.836988+0000","last_clean":"2024-09-22T03:23:09.836988+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:09.836988+0000","last_undegraded":"2024-09-22T03:23:09.836988+0000","last_fullsized":"2024-09-22T03:23:09.836988+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.837266+0000","last_peered":"2024-09-22T03:23:09.837266+0000","last_clean":"2024-09-22T03:23:09.837266+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:09.837266+0000","last_undegraded":"2024-09-22T03:23:09.837266+0000","last_fullsized":"2024-09-22T03:23:09.837266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'1","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841202+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.841202+0000","last_peered":"2024-09-22T03:23:09.841202+0000","last_clean":"2024-09-22T03:23:09.841202+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:09.841202+0000","last_undegraded":"2024-09-22T03:23:09.841202+0000","last_fullsized":"2024-09-22T03:23:09.841202+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.838867+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.838867+0000","last_peered":"2024-09-22T03:23:09.838867+0000","last_clean":"2024-09-22T03:23:09.838867+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:09.838867+0000","last_undegraded":"2024-09-22T03:23:09.838867+0000","last_fullsized":"2024-09-22T03:23:09.838867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842602+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:09.842602+0000","last_peered":"2024-09-22T03:23:09.842602+0000","last_clean":"2024-09-22T03:23:09.842602+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:09.842602+0000","last_undegraded":"2024-09-22T03:23:09.842602+0000","last_fullsized":"2024-09-22T03:23:09.842602+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842517+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:09.842517+0000","last_peered":"2024-09-22T03:23:09.842517+0000","last_clean":"2024-09-22T03:23:09.842517+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:09.842517+0000","last_undegraded":"2024-09-22T03:23:09.842517+0000","last_fullsized":"2024-09-22T03:23:09.842517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":151,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.835345+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:09.835345+0000","last_peered":"2024-09-22T03:23:09.835345+0000","last_clean":"2024-09-22T03:23:09.835345+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:09.835345+0000","last_undegraded":"2024-09-22T03:23:09.835345+0000","last_fullsized":"2024-09-22T03:23:09.835345+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1244,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296484,"statfs":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1632,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296096,"statfs":{"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":668,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297060,"statfs":{"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1568,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296160,"statfs":{"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:13.244 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T03:23:13.245 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-22T03:23:13.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:13.342+0000 7f465a957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:13.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:13.348+0000 7f465a957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:13.530 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 16 2024-09-22T03:23:13.530 INFO:teuthology.orchestra.run.smithi129.stdout:fsid 7f5a2dae-b56a-4afe-bb9b-fb3f65651791 2024-09-22T03:23:13.530 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-09-22T03:22:55.698211+0000 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:modified 2024-09-22T03:23:09.819356+0000 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:crush_version 3 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:full_ratio 0.95 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:backfillfull_ratio 0.9 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:nearfull_ratio 0.85 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:require_min_compat_client luminous 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:min_compat_client jewel 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:require_osd_release squid 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:stretch_mode_enabled false 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:allow_crimson=true 2024-09-22T03:23:13.531 INFO:teuthology.orchestra.run.smithi129.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 12 flags hashpspool,nopgchange,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2024-09-22T03:23:13.532 INFO:teuthology.orchestra.run.smithi129.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 16 flags hashpspool,nopgchange,selfmanaged_snaps,crimson stripe_width 0 application rbd read_balance_score 1.50 2024-09-22T03:23:13.532 INFO:teuthology.orchestra.run.smithi129.stdout: removed_snaps_queue [2~1] 2024-09-22T03:23:13.532 INFO:teuthology.orchestra.run.smithi129.stdout:max_osd 4 2024-09-22T03:23:13.532 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.129:6800/27947901 v2:172.21.15.129:6801/27947901 exists,up b141605f-6391-41b8-b680-cb81bf703b22 2024-09-22T03:23:13.532 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.129:6803/2887884686 v2:172.21.15.129:6804/2887884686 exists,up a1a59185-9c42-4359-af1f-3dd8be188aa7 2024-09-22T03:23:13.532 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.144:6804/3028982311 v2:172.21.15.144:6803/3028982311 exists,up c1b83060-4307-4966-a545-5476767a17c1 2024-09-22T03:23:13.532 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.144:6806/1383918015 v2:172.21.15.144:6807/1383918015 exists,up 487e7a88-d3c1-469d-83b4-222cb8676d02 2024-09-22T03:23:13.543 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.129:6800/27947901 v2:172.21.15.129:6801/27947901 exists,up b141605f-6391-41b8-b680-cb81bf703b22', 'osd.1 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.129:6803/2887884686 v2:172.21.15.129:6804/2887884686 exists,up a1a59185-9c42-4359-af1f-3dd8be188aa7', 'osd.2 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.144:6804/3028982311 v2:172.21.15.144:6803/3028982311 exists,up c1b83060-4307-4966-a545-5476767a17c1', 'osd.3 up in weight 1 up_from 9 up_thru 13 down_at 0 last_clean_interval [0,0) v2:172.21.15.144:6806/1383918015 v2:172.21.15.144:6807/1383918015 exists,up 487e7a88-d3c1-469d-83b4-222cb8676d02'] 2024-09-22T03:23:13.543 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-22T03:23:13.646 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mon.a", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster": "ceph", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "admin_socket_mode": "", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_client_required": "cephx, none", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_cluster_required": "cephx", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_debug": "true", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_service_required": "cephx", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-22T03:23:13.647 INFO:teuthology.orchestra.run.smithi129.stdout: "auth_supported": "cephx", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio": "true", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio_poll_ms": "250", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_aio_reap_max": "16", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_async_discard": "false", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_block_size": "4096", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_aio": "false", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_debug_inflight_ios": "false", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_enable_discard": "false", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_flock_retry": "3", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-22T03:23:13.648 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_inject_crash": "0", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_ioring": "false", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_ioring_hipri": "false", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bdev_type": "", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_alloc_size": "1048576", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_allocator": "hybrid", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_buffered_io": "true", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_check_for_zeros": "false", 2024-09-22T03:23:13.649 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_compact_log_sync": "false", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_max_log_runway": "4194304", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_max_prefetch": "1048576", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_min_flush_size": "524288", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_min_log_runway": "1048576", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_replay_recovery": "false", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-22T03:23:13.650 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_sync_write": "false", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_allocation_from_file": "true", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_allocator": "hybrid", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bdev_label_multi": "true", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-22T03:23:13.651 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_create": "true", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_db_create": "false", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_db_path": "", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_db_size": "0", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_path": "", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_preallocate_file": "false", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_size": "107374182400", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_wal_create": "false", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_wal_path": "", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_block_wal_size": "100663296", 2024-09-22T03:23:13.652 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs": "true", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_autotune": "true", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_size": "0", 2024-09-22T03:23:13.653 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_type": "2q", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_clone_cow": "true", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-22T03:23:13.654 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_mode": "none", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_csum_type": "crc32c", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_freelist": "false", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-22T03:23:13.655 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_small_allocations": "0", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_default_buffered_read": "true", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_default_buffered_write": "false", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-22T03:23:13.656 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fail_eio": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mount": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_umount": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-22T03:23:13.657 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_ignore_data_csum": "false", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_log_op_age": "5.000000", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_alloc_size": "0", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_blob_size": "0", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-22T03:23:13.658 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_max_deferred_txc": "32", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_min_alloc_size": "0", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_nid_prealloc": "1024", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_retry_disk_reads": "3", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_cf": "true", 2024-09-22T03:23:13.659 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_spdk_mem": "512", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-22T03:23:13.660 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_tracing": "false", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_zero_block_detection": "false", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-22T03:23:13.661 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_cluster_require_signatures": "false", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_cluster_require_version": "2", 2024-09-22T03:23:13.662 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_require_signatures": "false", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_require_version": "2", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_service_require_signatures": "false", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_service_require_version": "2", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "cephx_sign_messages": "true", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "chdir": "", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_acl_type": "", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_asio_thread_count": "2", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_cache_mid": "0.750000", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_cache_size": "16384", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_caps_release_delay": "5", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_check_pool_perm": "true", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_force_sync_read": "false", 2024-09-22T03:23:13.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_getattr_caps": "false", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_inject_features": "", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_debug_inject_tick_delay": "0", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_die_on_failed_remount": "false", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_dirsize_rbytes": "true", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_force_lazyio": "false", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_fs": "", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_inject_release_failure": "false", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_max_inline_size": "4096", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mds_namespace": "", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_metadata": "", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mount_gid": "-1", 2024-09-22T03:23:13.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mount_timeout": "300", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mount_uid": "-1", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_mountpoint": "/", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_notify_timeout": "10", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc": "true", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_max_dirty": "104857600", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_max_objects": "1000", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_size": "209715200", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_oc_target_dirty": "8388608", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_permissions": "true", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_quota": "true", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_quota_df": "true", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_readahead_max_bytes": "0", 2024-09-22T03:23:13.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_readahead_max_periods": "4", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_readahead_min": "131072", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_reconnect_stale": "false", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_shutdown_timeout": "30", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_snapdir": ".snap", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_tick_interval": "1", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_trace": "", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_try_dentry_invalidate": "false", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_use_faked_inos": "false", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_use_random_mds": "false", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_graylog": "false", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_graylog_port": "12201", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_monitors": "default=true", 2024-09-22T03:23:13.666 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_syslog": "false", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "clog_to_syslog_level": "info", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_addr": "-", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_network": "", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_network_interface": "", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zlib_isal": "false", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zlib_level": "5", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zlib_winsize": "-15", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "compressor_zstd_level": "1", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_alien_op_num_threads": "6", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-22T03:23:13.667 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_osd_stat_interval": "0", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_seastar_cpu_cores": "", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "crimson_seastar_num_threads": "0", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_location": "", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_location_hook": "", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_location_hook_timeout": "10", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "daemonize": "false", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "daos_pool": "tank", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_alienstore": "0/5", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_allow_any_pool_priority": "false", 2024-09-22T03:23:13.668 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_asok": "1/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_asok_assert_abort": "false", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_asserts_on_shutdown": "true", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_auth": "20/20", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_bdev": "1/3", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_bluefs": "1/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_bluestore": "1/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_buffer": "0/1", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_ceph_exporter": "1/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_cephfs_mirror": "0/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_cephsqlite": "0/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_client": "0/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_compressor": "1/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_context": "0/1", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_crimson_interrupt": "0/5", 2024-09-22T03:23:13.669 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_crush": "1/1", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_crypto": "1/5", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_cyanstore": "0/5", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_deliberately_leak_memory": "false", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_disable_randomized_ping": "false", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_dpdk": "1/5", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_eventtrace": "1/5", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_filer": "0/1", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_filestore": "1/3", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_finisher": "1/1", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_fuse": "1/5", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_heartbeat_testing_span": "0", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_heartbeatmap": "1/5", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-22T03:23:13.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_javaclient": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_journal": "1/3", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_journaler": "0/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_kstore": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_lockdep": "0/1", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mclock": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_balancer": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_locker": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_log": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_log_expire": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_migrator": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mds_quiesce": "3/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_memstore": "1/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mgr": "2/5", 2024-09-22T03:23:13.671 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mgrc": "1/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_mon": "20/20", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_monc": "0/10", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_ms": "1/1", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_none": "0/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_objclass": "0/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_objectcacher": "0/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_objecter": "0/1", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_optracker": "0/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_osd": "1/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_paxos": "20/20", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_perfcounter": "1/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_prioritycache": "1/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rados": "0/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd": "0/5", 2024-09-22T03:23:13.672 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd_mirror": "0/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd_pwl": "0/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rbd_replay": "0/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_refs": "0/0", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_reserver": "1/1", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw": "1/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_access": "1/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_datacache": "1/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_dbstore": "1/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_flight": "1/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rgw_sync": "1/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_rocksdb": "4/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore": "0/5", 2024-09-22T03:23:13.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_backref": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_cache": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_cleaner": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_device": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_epm": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_journal": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_lba": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_odata": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_omap": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_onode": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_t": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_seastore_tm": "0/5", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_striper": "0/1", 2024-09-22T03:23:13.674 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_test": "0/5", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_throttle": "1/1", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_timer": "0/1", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_tp": "0/5", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "debug_trace": "1/5", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "device_failure_prediction_mode": "none", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_graylog": "false", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_journald": "false", 2024-09-22T03:23:13.675 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_stderr": "true", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "err_to_syslog": "false", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event_tracing": "false", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_addr": "0.0.0.0", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_http_port": "9926", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_prio_limit": "5", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_sort_metrics": "true", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "exporter_stats_period": "5", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "fake_statfs_for_testing": "0", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "fatal_signal_handlers": "true", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "filer_max_purge_ops": "10", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "filer_max_truncate_ops": "128", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_apply_finisher_threads": "1", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_blackhole": "false", 2024-09-22T03:23:13.676 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_btrfs_clone_range": "true", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_btrfs_snap": "true", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_caller_concurrency": "10", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_collect_device_partition_information": "true", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_commit_timeout": "600.000000", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_debug_inject_read_err": "false", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_debug_omap_check": "false", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_debug_verify_split": "false", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_dump_file": "", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fadvise": "true", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fail_eio": "true", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fd_cache_shards": "16", 2024-09-22T03:23:13.677 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fd_cache_size": "128", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fiemap": "false", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fiemap_threshold": "4096", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_inject_stall": "0", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_journal_parallel": "false", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_journal_trailing": "false", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_journal_writeahead": "false", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_kill_at": "0", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-22T03:23:13.678 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs": "0", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size": "0", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_merge_threshold": "-10", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-22T03:23:13.679 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_odsync_write": "false", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_omap_backend": "rocksdb", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_omap_backend_path": "", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_op_thread_timeout": "60", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_op_threads": "2", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_punch_hole": "false", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-22T03:23:13.680 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_queue_max_ops": "50", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_seek_data_hole": "false", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_sloppy_crc": "false", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_splice": "false", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_split_multiple": "2", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_split_rand_factor": "20", 2024-09-22T03:23:13.681 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_update_to": "1000", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_enable": "true", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-22T03:23:13.682 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_xfs_extsize": "false", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "filestore_zfs_snap": "false", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fio_dir": "/tmp/fio", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fsid": "7f5a2dae-b56a-4afe-bb9b-fb3f65651791", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_allow_other": "true", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_atomic_o_trunc": "true", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_big_writes": "true", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_debug": "false", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_default_permissions": "false", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_disable_pagecache": "false", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_max_write": "0", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_multithreaded": "true", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_require_active_mds": "true", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_set_user_groups": "true", 2024-09-22T03:23:13.683 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_splice_move": "true", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_splice_read": "true", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_splice_write": "true", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "fuse_use_invalidate_cb": "true", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "gss_target_name": "ceph", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_file": "", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_inject_failure": "0", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "heartbeat_interval": "5", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "host": "smithi129", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-22T03:23:13.684 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "inject_early_sigterm": "false", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "jaeger_agent_port": "6799", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "jaeger_tracing_enable": "false", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_aio": "true", 2024-09-22T03:23:13.685 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_align_min_size": "65536", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_block_align": "true", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_block_size": "4096", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_dio": "true", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_discard": "false", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_force_aio": "false", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_ignore_corruption": "false", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_max_write_bytes": "10485760", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_max_write_entries": "100", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_replay_from": "0", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-22T03:23:13.686 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_write_header_frequency": "0", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "journal_zero_on_create": "false", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "journaler_prefetch_periods": "10", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "journaler_prezero_periods": "5", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "journaler_write_head_interval": "15", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "key": "", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "keyfile": "", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_backend": "rocksdb", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_default_stripe_size": "65536", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_fsck_on_mount": "false", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_max_bytes": "67108864", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_max_ops": "512", 2024-09-22T03:23:13.687 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_nid_prealloc": "1024", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_onode_map_size": "1024", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_sync_submit_transaction": "false", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "kstore_sync_transaction": "false", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "librados_thread_count": "2", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "lockdep": "false", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "lockdep_force_backtrace": "false", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "log_coarse_timestamps": "true", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "log_flush_on_exit": "false", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "log_graylog_host": "127.0.0.1", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "log_graylog_port": "12201", 2024-09-22T03:23:13.688 INFO:teuthology.orchestra.run.smithi129.stdout: "log_max_new": "1000", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_max_recent": "10000", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_stderr_prefix": "", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_stop_at_utilization": "0.970000", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": "true", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_graylog": "false", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_journald": "false", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_stderr": "false", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_syslog": "false", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "max_rotating_auth_attempts": "10", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_action_on_write_error": "1", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_alternate_name_max": "8192", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_asio_thread_count": "2", 2024-09-22T03:23:13.689 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_export_pin": "true", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_dirs": "true", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_interval": "5", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_interval": "10", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_max": "-1", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_max_until": "-1", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_merge_size": "50", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_midchunk": "0.300000", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_min_start": "0.200000", 2024-09-22T03:23:13.690 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_minchunk": "0.001000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_mode": "0", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_need_max": "1.200000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_need_min": "0.800000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_overload_epochs": "2", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_bits": "3", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_size": "10000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_target_decay": "10.000000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-22T03:23:13.691 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_beacon_grace": "15.000000", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_beacon_interval": "4.000000", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_mid": "0.700000", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_delay": "0", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_release_free_interval": "10", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_reservation": "0.050000", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_trim_interval": "1", 2024-09-22T03:23:13.692 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache_trim_threshold": "262144", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_client_prealloc_inos": "1000", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_connect_bootstrapping": "false", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_damage_table_max_entries": "10000", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_auth_pins": "false", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_frag": "false", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_scatterstat": "false", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_debug_subtrees": "false", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_decay_halflife": "5.000000", 2024-09-22T03:23:13.693 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_default_dir_hash": "2", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_defer_session_stale": "true", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_deny_all_reconnect": "false", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_keys_per_op": "16384", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_max_commit_size": "10", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_max_entries": "0", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dir_prefetch": "true", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_on_map": "false", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_early_reply": "true", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_enable_op_tracker": "true", 2024-09-22T03:23:13.694 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_enforce_unique_name": "true", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_random": "true", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_health_summarize_threshold": "10", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-22T03:23:13.695 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_health_dummy": "false", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_migrator_session_race": "false", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_join_fs": "", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_journal_format": "1", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_create_at": "0", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_export_at": "0", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_import_at": "0", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_journal_at": "0", 2024-09-22T03:23:13.696 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_journal_expire_at": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_journal_replay_at": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_link_at": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_mdstable_at": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_openc_at": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_rename_at": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_kill_shutdown_at": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_event_large_threshold": "524288", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_events_per_segment": "1024", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_max_events": "-1", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_max_segments": "128", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_pause": "false", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_segment_size": "0", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-22T03:23:13.697 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_trim_threshold": "128", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log_warn_factor": "2.000000", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_caps_per_client": "1048576", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_completed_flushes": "100000", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_completed_requests": "100000", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_export_size": "20971520", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_file_recover": "32", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_purge_files": "64", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_purge_ops": "8192", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-22T03:23:13.698 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_max_snaps_per_dir": "100", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_metrics_update_interval": "2", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_min_caps_per_client": "100", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_min_caps_working_set": "10000", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_numa_node": "-1", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_complaint_time": "30.000000", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_duration": "600", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_size": "20", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_slow_op_size": "20", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_op_log_threshold": "5", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_ping_grace": "15", 2024-09-22T03:23:13.699 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_ping_interval": "5", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_max_caps": "30000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_recall_warning_threshold": "262144", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_replay_interval": "1.000000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_root_ino_gid": "0", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_root_ino_uid": "0", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-22T03:23:13.700 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_shutdown_check": "0", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_skip_ino": "0", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_snap_max_uid": "4294967294", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_snap_min_uid": "0", 2024-09-22T03:23:13.701 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_snap_rstat": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_standby_replay_damaged": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_symlink_recovery": "true", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_thrash_exports": "0", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_thrash_fragments": "0", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_tick_interval": "5.000000", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_valgrind_exit": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_verify_backtrace": "1", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_verify_scatter": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_wipe_sessions": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool_debug": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-22T03:23:13.702 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_device_bytes": "1073741824", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_page_set": "false", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "memstore_page_size": "65536", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_client_bytes": "134217728", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_client_messages": "512", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_disabled_modules": "", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_enable_op_tracker": "true", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_max_pg_num_change": "128", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mds_bytes": "134217728", 2024-09-22T03:23:13.703 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mds_messages": "128", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mon_bytes": "134217728", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_mon_messages": "128", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_duration": "600", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_size": "20", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_op_log_threshold": "5", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_osd_bytes": "536870912", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_osd_messages": "8192", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_pool": "true", 2024-09-22T03:23:13.704 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_standby_modules": "true", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_stats_period": "5", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_stats_threshold": "5", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_tick_period": "2", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_allow_pool_delete": "true", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_allow_pool_size_one": "true", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_auth_validate_all_caps": "true", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_bytes": "104857600", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_directed_command_retry": "2", 2024-09-22T03:23:13.705 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_hunt_parallel": "3", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_log_interval": "1.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_ping_interval": "10.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_client_target_rank": "-1", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_file_level": "debug", 2024-09-22T03:23:13.706 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_file": "true", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_journald": "false", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_compact_on_bootstrap": "false", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_compact_on_start": "false", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_compact_on_trim": "true", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-22T03:23:13.707 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_cpu_threads": "4", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_crush_min_required_version": "hammer", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_daemon_bytes": "419430400", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data_avail_crit": "5", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data_avail_warn": "5", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_data_size_warn": "16106127360", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_dump_json": "false", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_dump_transactions": "false", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_extra_checks": "false", 2024-09-22T03:23:13.708 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_require_reef": "false", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_no_require_squid": "false", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_down_mkfs_grace": "120", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_down_uptime_grace": "60", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_election_default_strategy": "1", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_election_timeout": "5.000000", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_elector_ping_divisor": "2", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-22T03:23:13.709 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_enable_op_tracker": "true", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_fake_pool_delete": "false", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_force_quorum_join": "false", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_globalid_prealloc": "10000", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_detail_to_clog": "false", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_log_update_period": "5", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_max_detail": "50", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_to_clog": "true", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_to_clog_interval": "600", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_host": "172.21.15.129", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_host_override": "", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_initial_members": "", 2024-09-22T03:23:13.710 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_lease": "5.000000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_log_full_interval": "50", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_log_max": "10000", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_log_max_summary": "50", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_log_epochs": "500", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-22T03:23:13.711 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_osd": "10000", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_pg_per_osd": "10000", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_pool_pg_num": "65536", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mds_force_trim_to": "0", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mds_skip_sanity": "false", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_memory_autotune": "true", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_memory_target": "2147483648", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_beacon_grace": "30", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_digest_period": "5", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_inactive_grace": "60", 2024-09-22T03:23:13.712 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_min_osdmap_epochs": "50", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_complaint_time": "30", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_duration": "600", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_size": "20", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_slow_op_size": "20", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_op_log_threshold": "5", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_auto_mark_in": "false", 2024-09-22T03:23:13.713 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_cache_size": "500", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_down_out_interval": "600", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_force_trim_to": "0", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-22T03:23:13.714 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_min_down_reporters": "2", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_report_timeout": "900", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-22T03:23:13.715 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_stuck_threshold": "60", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_probe_timeout": "2.000000", 2024-09-22T03:23:13.716 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_max_change": "0.050000", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_max_osds": "4", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_interval": "86400", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_max_keys": "100", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_scrub_timeout": "300", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_session_timeout": "300", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_smart_report_timeout": "5", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stat_smooth_intervals": "6", 2024-09-22T03:23:13.717 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_pool_min_size": "2", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_pool_size": "4", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_debug": "false", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_provider_kill_at": "0", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_requester_kill_at": "0", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_sync_timeout": "60.000000", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_target_pg_per_osd": "100", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_tick_interval": "5", 2024-09-22T03:23:13.718 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_timecheck_interval": "300.000000", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_use_min_delay_socket": "false", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_older_version_delay": "604800", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_misplaced": "false", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-22T03:23:13.719 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_older_version": "true", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "monmap": "", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T03:23:13.720 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "motr_tracing_enabled": "false", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_op_threads": "3", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_cm": "false", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_device_name": "", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_dscp": "96", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_local_gid": "", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_port_num": "1", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-22T03:23:13.721 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_sl": "3", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_support_srq": "true", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_rdma_type": "ib", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_async_reap_threshold": "5", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_before_connect": "false", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_ipv4": "true", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_ipv6": "false", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_msgr1": "true", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_msgr2": "true", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_port_max": "7568", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_port_min": "6800", 2024-09-22T03:23:13.722 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_retry_count": "3", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_bind_retry_delay": "5", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_client": "false", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_mds": "false", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_mgr": "false", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_mon": "false", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_blackhole_osd": "false", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_client_mode": "crc", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_cluster_mode": "crc", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_cluster_type": "", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_compress_secure": "false", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_connection_idle_timeout": "900", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_connection_ready_timeout": "10", 2024-09-22T03:23:13.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_crc_data": "true", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_crc_header": "true", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_bad_msg": "false", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_bug": "true", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_old_message": "true", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_skipped_message": "false", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_coremask": "0xF", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_enable_tso": "true", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-22T03:23:13.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_hugepages": "", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_lro": "true", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_memory_channel": "4", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_pmd": "", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_port_id": "0", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-22T03:23:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_dump_on_send": "false", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_initial_backoff": "0.200000", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_delay_max": "1.000000", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_delay_type": "", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_network_congestion": "0", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_inject_socket_failures": "0", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_learn_addr_from_peer": "true", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_max_accept_failures": "4", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_max_backoff": "15.000000", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_mon_client_mode": "crc", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_mon_cluster_mode": "crc", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_mon_service_mode": "crc", 2024-09-22T03:23:13.726 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_osd_compress_min_size": "1024", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_osd_compress_mode": "none", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_pq_min_cost": "65536", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_public_type": "", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_service_mode": "crc", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_listen_backlog": "512", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_nodelay": "true", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_tcp_rcvbuf": "0", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ms_type": "async+posix", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "no_config_file": "false", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_completion_locks_per_session": "32", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-22T03:23:13.727 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_inflight_ops": "1024", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_tick_interval": "5.000000", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter_timeout": "10.000000", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "objectstore_blackhole": "false", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "openssl_engine_opts": "", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_delay_time": "5.000000", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_hist_halflife": "1000", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_max_low_ops": "2", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_max_ops": "4", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-22T03:23:13.728 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_agent_slop": "0.020000", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_asio_thread_count": "2", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_async_recovery_min_cost": "100", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backfill_scan_max": "512", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backfill_scan_min": "64", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backoff_on_degraded": "false", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backoff_on_peering": "false", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_backoff_on_unfound": "true", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_beacon_report_interval": "300", 2024-09-22T03:23:13.729 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_duration": "30", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_max_block_size": "67108864", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_blkin_trace_all": "false", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_check_for_log_corruption": "false", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T03:23:13.730 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_class_update_on_start": "true", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_message_cap": "256", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_message_size_cap": "524288000", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_op_priority": "63", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_client_watch_timeout": "30", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_command_max_records": "256", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_compact_on_start": "false", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_update_on_start": "true", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_crush_update_weight_set": "true", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-22T03:23:13.731 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_feed_pullee": "-1", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_misdirected_ops": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_no_acting_change": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_no_purge_strays": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_op_order": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-22T03:23:13.732 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_shutdown": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_trim_objects": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_snaps": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_keys": "1024", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-22T03:23:13.733 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_stride": "524288", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_default_notify_timeout": "30", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep": "0.000000", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_discard_disconnected_ops": "true", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_enable_op_tracker": "true", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-22T03:23:13.734 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_info": "true", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_shutdown": "true", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_function_tracing": "false", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_grace": "20", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_interval": "6", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_min_peers": "10", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_min_size": "2000", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_stale": "600", 2024-09-22T03:23:13.735 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_hit_set_max_size": "100000", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_hit_set_min_size": "1000", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_journal_size": "5120", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_kill_backfill_at": "0", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_cache_size": "50", 2024-09-22T03:23:13.736 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_dedup": "true", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_message_max": "40", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_map_share_max_epochs": "40", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_attr_name_len": "100", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_attr_size": "0", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_backfills": "1", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_markdown_count": "5", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_markdown_period": "600", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_object_name_len": "2048", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_object_namespace_len": "256", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_object_size": "134217728", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-22T03:23:13.737 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pg_blocked_by": "16", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pg_log_entries": "10000", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_pgls": "1024", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_push_cost": "8388608", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_push_objects": "10", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_scrubs": "3", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_trimming_pgs": "2", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_write_op_reply_len": "64", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_max_write_size": "90", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-22T03:23:13.738 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_profile": "balanced", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-22T03:23:13.739 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_base": "805306368", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_cache_min": "134217728", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_target": "4294967296", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_target_autotune": "false", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_min_pg_log_entries": "250", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_min_recovery_priority": "0", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_report_interval": "5", 2024-09-22T03:23:13.740 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_num_cache_shards": "32", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_num_op_tracker_shard": "32", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_numa_auto_affinity": "true", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_numa_node": "-1", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_numa_prefer_iface": "true", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objecter_finishers": "1", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objectstore": "bluestore", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objectstore_fuse": "false", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_objectstore_tracing": "false", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_complaint_time": "30.000000", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_duration": "600", 2024-09-22T03:23:13.741 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_size": "20", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_slow_op_size": "20", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_log_threshold": "5", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_shards": "0", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_shards_hdd": "5", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_shards_ssd": "8", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_pq_min_cost": "65536", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_queue_cut_off": "high", 2024-09-22T03:23:13.742 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_op_thread_timeout": "15", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_open_classes_on_start": "true", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_os_flags": "0", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_peering_op_priority": "255", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_delete_cost": "1048576", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_delete_priority": "5", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_log_trim_max": "10000", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_log_trim_min": "100", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-22T03:23:13.743 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_crimson": "true", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-22T03:23:13.744 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_flags": "0", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_min_size": "2", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_pg_num": "32", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_pgp_num": "0", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_read_ratio": "70", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_size": "3", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_default_type": "replicated", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-22T03:23:13.745 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_push_per_object_cost": "1000", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recover_clone_overlap": "true", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_cost": "20971520", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_active": "0", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_max_single_start": "1", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_op_priority": "3", 2024-09-22T03:23:13.746 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_priority": "5", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep": "0.000000", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_repair_during_recovery": "false", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_requested_scrub_priority": "120", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_auto_repair": "false", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_begin_hour": "0", 2024-09-22T03:23:13.747 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_begin_week_day": "0", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_chunk_max": "25", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_chunk_min": "5", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_cost": "52428800", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_during_recovery": "false", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_end_hour": "0", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_end_week_day": "0", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_event_cost": "4096", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_invalid_stats": "true", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_max_preemptions": "5", 2024-09-22T03:23:13.748 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_priority": "5", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_scrub_sleep": "0.000000", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_skip_data_digest": "false", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_smart_report_timeout": "5", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_cost": "1048576", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_priority": "5", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-22T03:23:13.749 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_target_transaction_size": "30", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_tracing": "false", 2024-09-22T03:23:13.750 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_use_stale_snap": "false", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "osdc_blkin_trace_all": "false", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_kill_at": "0", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_max_join_drift": "10", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_min": "500", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_min_wait": "0.050000", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_propose_interval": "1.000000", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_service_trim_max": "500", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_service_trim_min": "10", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_stash_full_interval": "25", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_trim_max": "500", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "paxos_trim_min": "250", 2024-09-22T03:23:13.751 INFO:teuthology.orchestra.run.smithi129.stdout: "perf": "true", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addr": "-", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "public_addrv": "", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "public_bind_addr": "-", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "public_network": "", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "public_network_interface": "", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "qat_compressor_busy_polling": "false", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "qat_compressor_enabled": "false", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "qat_compressor_session_max_number": "256", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "rados_mon_op_timeout": "0", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "rados_osd_op_timeout": "0", 2024-09-22T03:23:13.752 INFO:teuthology.orchestra.run.smithi129.stdout: "rados_tracing": "false", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_atime_update_interval": "60", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_balance_parent_reads": "false", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_balance_snap_reads": "false", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_blkin_trace_all": "false", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache": "true", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_policy": "writearound", 2024-09-22T03:23:13.753 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_size": "33554432", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_clone_copy_on_read": "false", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_compression_hint": "none", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_concurrent_management_ops": "10", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_clone_format": "auto", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_data_pool": "", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_features": "61", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_format": "2", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_map_options": "", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_order": "22", 2024-09-22T03:23:13.754 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_pool": "rbd", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_stripe_count": "0", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_default_stripe_unit": "0", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_enable_alloc_hint": "true", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_io_scheduler": "simple", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-22T03:23:13.755 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_order": "24", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_pool": "", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_journal_splay_width": "4", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_localize_parent_reads": "false", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_localize_snap_reads": "false", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-22T03:23:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-22T03:23:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_mtime_update_interval": "60", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_non_blocking_aio": "true", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_op_thread_timeout": "60", 2024-09-22T03:23:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_op_threads": "1", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_parent_cache_enabled": "false", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_plugins": "", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_bps_burst": "0", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_bps_limit": "0", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_exclude_ops": "0", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_iops_burst": "0", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_iops_limit": "0", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-22T03:23:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_read_from_replica_policy": "default", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-22T03:23:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_skip_partial_discard": "true", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_tracing": "false", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_validate_names": "true", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_validate_pool": "true", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "restapi_base_url": "", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "restapi_log_level": "", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_acl_grants_max_num": "100", 2024-09-22T03:23:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_admin_entry": "admin", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_backend_store": "rados", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_barbican_url": "", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_beast_enable_async": "true", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_counters_cache": "false", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-22T03:23:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cache_enabled": "true", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cache_expiry_interval": "900", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cache_lru_size": "25000", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_config_store": "rados", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_content_length_compat": "false", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_copy_obj_progress": "true", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cors_rules_max_num": "100", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_cross_domain_policy": "", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_addr": "", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-22T03:23:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_password": "", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_kmip_username": "", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_require_ssl": "true", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-22T03:23:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_addr": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_auth": "token", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_namespace": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_prefix": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_token_file": "", 2024-09-22T03:23:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_buffersize": "524288", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_low_speed_time": "300", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-22T03:23:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_d4n_port": "6379", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_changes_size": "1000", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_num_shards": "128", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_log_window": "30", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-22T03:23:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-22T03:23:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dns_name": "", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dns_s3website_name": "", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_dynamic_resharding": "true", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_gc_threads": "true", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_lc_threads": "true", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_ops_log": "false", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_quota_threads": "true", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_static_website": "false", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enable_usage_log": "false", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_enforce_swift_acls": "true", 2024-09-22T03:23:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_exit_timeout_secs": "120", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_expose_bucket": "false", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_extended_http_attrs": "", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_filter": "none", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_frontends": "beast port=7480", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_deferred": "50", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_objs": "32", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_gc_processor_period": "3600", 2024-09-22T03:23:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_init_timeout": "300", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_kafka_connection_idle": "300", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_domain": "", 2024-09-22T03:23:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_password": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_password_path": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_project": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_tenant": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_token": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_token_path": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_admin_user": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_api_version": "2", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_domain": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_password": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_project": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_barbican_user": "", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-22T03:23:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_url": "", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_debug_interval": "-1", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_lock_max_time": "90", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_objs": "32", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_rules": "1000", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_worker": "3", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lc_thread_delay": "0", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_dnattr": "uid", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-22T03:23:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_searchfilter": "", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ldap_uri": "", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_http_headers": "", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_log_object_name_utc": "false", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_attr_name_len": "0", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_attr_size": "0", 2024-09-22T03:23:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_chunk_size": "4194304", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_listing_results": "1000", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_notify_retries": "10", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_put_param_size": "1048576", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_put_size": "5368709120", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_max_slo_entries": "1000", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_md_log_max_shards": "64", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-22T03:23:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_mp_lock_max_time": "600", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_max_gc": "300", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-22T03:23:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_num_async_rados_threads": "32", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_num_control_oids": "8", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_numa_node": "-1", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_objexp_chunk_size": "100", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_objexp_gc_interval": "600", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_op_thread_timeout": "600", 2024-09-22T03:23:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_op_tracing": "false", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_opa_token": "", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_opa_url": "", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_opa_verify_ssl": "true", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_rados": "false", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_ops_log_socket_path": "", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_push_interval": "2.000000", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-22T03:23:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_lanes": "3", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_cache_partitions": "3", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_print_continue": "true", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rados_tracing": "false", 2024-09-22T03:23:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_realm": "", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_realm_id": "", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_region": "", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_request_uri": "", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_batch_size": "64", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_max_aio": "128", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_num_logs": "16", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_reshard_thread_interval": "600", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_resolve_cname": "false", 2024-09-22T03:23:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_run_sync_thread": "true", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_scheduler_type": "throttler", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_script_uri": "", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_service_provider_name": "", 2024-09-22T03:23:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_client_id": "", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_client_secret": "", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_entry": "sts", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_key": "sts", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_min_session_duration": "900", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sts_token_introspection_url": "", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_account_in_url": "false", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_auth_entry": "auth", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_auth_url": "", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_custom_header": "", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_need_stats": "true", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_tenant_name": "", 2024-09-22T03:23:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_token_expiration": "86400", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_url": "", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_url_prefix": "swift", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_lease_period": "120", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-22T03:23:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_thread_pool_size": "512", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_comment": "", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_createby": "", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_encoding": "", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_flag": "false", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_origin": "", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_torrent_tracker": "", 2024-09-22T03:23:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_trust_forwarded_https": "false", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_max_shards": "32", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_usage_max_user_shards": "1", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_use_opa_authz": "false", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_counters_cache": "false", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_max_buckets": "1000", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_policies_max_num": "100", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-22T03:23:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_user_unique_email": "true", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_verify_ssl": "true", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zone": "", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zone_id": "", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zonegroup": "", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zonegroup_id": "", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgwlc_auto_session_clear": "true", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_block_size": "4096", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-22T03:23:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_size": "536870912", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_index_type": "binary_search", 2024-09-22T03:23:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_partition_filters": "false", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_perf": "false", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "run_dir": "/var/run/ceph", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_block_create": "true", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_cache_lru_size": "67108864", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_default_max_object_size": "16777216", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_device_size": "53687091200", 2024-09-22T03:23:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_full_integrity_check": "false", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_batch_capacity": "16", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_main_device_type": "SSD", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "seastore_segment_size": "67108864", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "set_keepcaps": "false", 2024-09-22T03:23:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "setgroup": "", 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "setuser": "", 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "setuser_match_path": "", 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "thp": "false", 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "threadpool_default_timeout": "60", 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "throttler_perf_counter": "true" 2024-09-22T03:23:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-22T03:23:13.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-22T03:23:13.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:13.891+0000 7f2566997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:13.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:13.895+0000 7f2566997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:13.909 INFO:teuthology.orchestra.run.smithi129.stdout:{} 2024-09-22T03:23:13.909 INFO:teuthology.orchestra.run.smithi129.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T03:23:13.920 DEBUG:teuthology.orchestra.run.smithi129:> type ceph-objectstore-tool 2024-09-22T03:23:13.934 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T03:23:13.934 DEBUG:teuthology.orchestra.run.smithi144:> type ceph-objectstore-tool 2024-09-22T03:23:13.952 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T03:23:13.953 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-22T03:23:13.962 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:23:13.963 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:23:13.964 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-22T03:23:13.964 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-22T03:23:13.964 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-22T03:23:13.965 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:13.965 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:13.965 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:13.965 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-22T03:23:13.965 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:14.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:14.080+0000 7fa0f866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:14.080+0000 7f52c3d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:14.084+0000 7fa0f866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:14.080+0000 7ff334917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:14.087+0000 7f52c3d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:14.091+0000 7ff334917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:14.275 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:14.276 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:14.288 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-22T03:23:13.937830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":5112,"kb_used_data":5112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209185800,"statfs":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.995236"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841697+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:09.841697+0000","last_peered":"2024-09-22T03:23:09.841697+0000","last_clean":"2024-09-22T03:23:09.841697+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:09.841697+0000","last_undegraded":"2024-09-22T03:23:09.841697+0000","last_fullsized":"2024-09-22T03:23:09.841697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837288+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:09.837288+0000","last_peered":"2024-09-22T03:23:09.837288+0000","last_clean":"2024-09-22T03:23:09.837288+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:09.837288+0000","last_undegraded":"2024-09-22T03:23:09.837288+0000","last_fullsized":"2024-09-22T03:23:09.837288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.836988+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.836988+0000","last_peered":"2024-09-22T03:23:09.836988+0000","last_clean":"2024-09-22T03:23:09.836988+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:09.836988+0000","last_undegraded":"2024-09-22T03:23:09.836988+0000","last_fullsized":"2024-09-22T03:23:09.836988+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.837266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:09.837266+0000","last_peered":"2024-09-22T03:23:09.837266+0000","last_clean":"2024-09-22T03:23:09.837266+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:09.837266+0000","last_undegraded":"2024-09-22T03:23:09.837266+0000","last_fullsized":"2024-09-22T03:23:09.837266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'1","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.841202+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.841202+0000","last_peered":"2024-09-22T03:23:09.841202+0000","last_clean":"2024-09-22T03:23:09.841202+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:09.841202+0000","last_undegraded":"2024-09-22T03:23:09.841202+0000","last_fullsized":"2024-09-22T03:23:09.841202+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.838867+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:09.838867+0000","last_peered":"2024-09-22T03:23:09.838867+0000","last_clean":"2024-09-22T03:23:09.838867+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:09.838867+0000","last_undegraded":"2024-09-22T03:23:09.838867+0000","last_fullsized":"2024-09-22T03:23:09.838867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842602+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:09.842602+0000","last_peered":"2024-09-22T03:23:09.842602+0000","last_clean":"2024-09-22T03:23:09.842602+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:09.842602+0000","last_undegraded":"2024-09-22T03:23:09.842602+0000","last_fullsized":"2024-09-22T03:23:09.842602+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.842517+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:09.842517+0000","last_peered":"2024-09-22T03:23:09.842517+0000","last_clean":"2024-09-22T03:23:09.842517+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:09.842517+0000","last_undegraded":"2024-09-22T03:23:09.842517+0000","last_fullsized":"2024-09-22T03:23:09.842517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":151,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-22T03:23:09.835345+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:09.835345+0000","last_peered":"2024-09-22T03:23:09.835345+0000","last_clean":"2024-09-22T03:23:09.835345+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:09.835345+0000","last_undegraded":"2024-09-22T03:23:09.835345+0000","last_fullsized":"2024-09-22T03:23:09.835345+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214206259200,"internally_reserved":0,"allocated":5234688,"data_stored":5234688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1244,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296484,"statfs":{"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1632,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296096,"statfs":{"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":668,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297060,"statfs":{"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1568,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296160,"statfs":{"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551267840,"internally_reserved":0,"allocated":1605632,"data_stored":1605632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53552189440,"internally_reserved":0,"allocated":684032,"data_stored":684032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53551202304,"internally_reserved":0,"allocated":1671168,"data_stored":1671168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53551599616,"internally_reserved":0,"allocated":1273856,"data_stored":1273856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:14.963 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_0' created 2024-09-22T03:23:14.980 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-22T03:23:15.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:15.090+0000 7fc4f250e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:15.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:15.102+0000 7fc4f250e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:15.966 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-22T03:23:16.162 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:16.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:16.261+0000 7f3eebd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:16.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:16.266+0000 7f3eebd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:16.986 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-22T03:23:16.989 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:23:16.989 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:23:17.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:23:17.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:23:17.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_29655 2024-09-22T03:23:17.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:23:17.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:23:18.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 217 201 1.57023 1.57031 0.0254259 0.0766713 2024-09-22T03:23:18.334 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:18.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:18.442+0000 7fdb0984f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:18.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:18.447+0000 7fdb0984f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 431 415 1.62099 1.67188 0.0572619 0.0758903 2024-09-22T03:23:19.289 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:19.290 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:19.290 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:23:19.290 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:23:19.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.438+0000 7f42b40a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.447+0000 7f42b40a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:19.467 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:19.467 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:19.470 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:19.471 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:19.471 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:23:19.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.602+0000 7fa2e6f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.610+0000 7fa2e6f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:19.647 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:19.648 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:19.648 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:19.651 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:19.652 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:19.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:23:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.801+0000 7f39e7557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.812+0000 7f39e7557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:19.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:19.828 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:19.829 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:19.830 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:19.830 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:23:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.974+0000 7f59d9b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:19.983+0000 7f59d9b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:20.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:20.001 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:20.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 639 623 1.62228 1.625 0.0205887 0.0751675 2024-09-22T03:23:20.554 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:20.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:20.677+0000 7f47e874e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:20.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:20.682+0000 7f47e874e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:21.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 841 825 1.61121 1.57812 0.107724 0.0766499 2024-09-22T03:23:22.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 1041 1025 1.60144 1.5625 0.135535 0.0768985 2024-09-22T03:23:22.794 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:22.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:22.914+0000 7ff57146c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:22.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:22.920+0000 7ff57146c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:23.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 1254 1238 1.61186 1.66406 0.0240146 0.0767113 2024-09-22T03:23:24.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 1459 1443 1.61037 1.60156 0.0317342 0.0768127 2024-09-22T03:23:25.002 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:25.003 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:23:25.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 15 1646 1631 1.59265 1.46875 0.0488523 0.0778073 2024-09-22T03:23:25.118 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:25.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:25.121+0000 7f705d957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:25.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:25.133+0000 7f705d957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:25.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:25.264+0000 7f8b1694e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:25.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:25.272+0000 7f8b1694e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:25.985 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:23:26.002 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:23:26.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1832 1816 1.57626 1.44531 0.0948715 0.0789723 2024-09-22T03:23:26.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:26.146+0000 7f4b11197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:26.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:26.157+0000 7f4b11197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:26.991 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:23:27.014 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:23:27.046 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 2021 2005 1.56628 1.47656 0.0385575 0.079041 2024-09-22T03:23:27.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:27.124+0000 7f6879e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:27.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:27.135+0000 7f6879e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:27.538 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:27.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:27.651+0000 7fa149557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:27.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:27.656+0000 7fa149557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:27.997 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:23:28.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:23:28.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 2232 2216 1.57374 1.64844 0.02608 0.0791154 2024-09-22T03:23:28.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:28.141+0000 7f8f76b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:28.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:28.151+0000 7f8f76b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:29.004 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:23:29.028 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:23:29.028 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:29.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 2434 2418 1.57409 1.57812 0.0519762 0.079048 2024-09-22T03:23:29.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:29.161+0000 7f4f6746c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:29.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:29.169+0000 7f4f6746c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:29.395 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:29.411 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-22T03:23:29.004172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16925235,"num_objects":255,"num_object_clones":0,"num_object_copies":765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2133,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2082,"ondisk_log_size":2082,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":103724,"kb_used_data":103724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209087188,"statfs":{"total":214211493888,"available":214105280512,"internally_reserved":0,"allocated":106213376,"data_stored":106213376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7938048,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":969,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.064298"},"pg_stats":[{"pgid":"3.f","version":"20'143","reported_seq":159,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.400431+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:23:26.400431+0000","last_peered":"2024-09-22T03:23:26.400431+0000","last_clean":"2024-09-22T03:23:26.400431+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:23:26.400431+0000","last_undegraded":"2024-09-22T03:23:26.400431+0000","last_fullsized":"2024-09-22T03:23:26.400431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"20'135","reported_seq":151,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.738054+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:23:26.738054+0000","last_peered":"2024-09-22T03:23:26.738054+0000","last_clean":"2024-09-22T03:23:26.738054+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:23:26.738054+0000","last_undegraded":"2024-09-22T03:23:26.738054+0000","last_fullsized":"2024-09-22T03:23:26.738054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'111","reported_seq":127,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.439217+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:23:26.439217+0000","last_peered":"2024-09-22T03:23:26.439217+0000","last_clean":"2024-09-22T03:23:26.439217+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:23:26.439217+0000","last_undegraded":"2024-09-22T03:23:26.439217+0000","last_fullsized":"2024-09-22T03:23:26.439217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"19'104","reported_seq":119,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.037256+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:23:26.037256+0000","last_peered":"2024-09-22T03:23:26.037256+0000","last_clean":"2024-09-22T03:23:26.037256+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:23:26.037256+0000","last_undegraded":"2024-09-22T03:23:26.037256+0000","last_fullsized":"2024-09-22T03:23:26.037256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'119","reported_seq":135,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.523856+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:23:26.523856+0000","last_peered":"2024-09-22T03:23:26.523856+0000","last_clean":"2024-09-22T03:23:26.523856+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:23:26.523856+0000","last_undegraded":"2024-09-22T03:23:26.523856+0000","last_fullsized":"2024-09-22T03:23:26.523856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"19'96","reported_seq":111,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.035054+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:23:26.035054+0000","last_peered":"2024-09-22T03:23:26.035054+0000","last_clean":"2024-09-22T03:23:26.035054+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:23:26.035054+0000","last_undegraded":"2024-09-22T03:23:26.035054+0000","last_fullsized":"2024-09-22T03:23:26.035054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"20'167","reported_seq":183,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.792217+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:23:26.792217+0000","last_peered":"2024-09-22T03:23:26.792217+0000","last_clean":"2024-09-22T03:23:26.792217+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:23:26.792217+0000","last_undegraded":"2024-09-22T03:23:26.792217+0000","last_fullsized":"2024-09-22T03:23:26.792217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"19'112","reported_seq":127,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.005336+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:23:26.005336+0000","last_peered":"2024-09-22T03:23:26.005336+0000","last_clean":"2024-09-22T03:23:26.005336+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:23:26.005336+0000","last_undegraded":"2024-09-22T03:23:26.005336+0000","last_fullsized":"2024-09-22T03:23:26.005336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.001626+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:26.001626+0000","last_peered":"2024-09-22T03:23:26.001626+0000","last_clean":"2024-09-22T03:23:26.001626+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:26.001626+0000","last_undegraded":"2024-09-22T03:23:26.001626+0000","last_fullsized":"2024-09-22T03:23:26.001626+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":29,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.009152+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:26.009152+0000","last_peered":"2024-09-22T03:23:26.009152+0000","last_clean":"2024-09-22T03:23:26.009152+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:26.009152+0000","last_undegraded":"2024-09-22T03:23:26.009152+0000","last_fullsized":"2024-09-22T03:23:26.009152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"19'88","reported_seq":103,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.008412+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:23:26.008412+0000","last_peered":"2024-09-22T03:23:26.008412+0000","last_clean":"2024-09-22T03:23:26.008412+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:23:26.008412+0000","last_undegraded":"2024-09-22T03:23:26.008412+0000","last_fullsized":"2024-09-22T03:23:26.008412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.040503+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:26.040503+0000","last_peered":"2024-09-22T03:23:26.040503+0000","last_clean":"2024-09-22T03:23:26.040503+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:26.040503+0000","last_undegraded":"2024-09-22T03:23:26.040503+0000","last_fullsized":"2024-09-22T03:23:26.040503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"20'119","reported_seq":135,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.291408+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:23:26.291408+0000","last_peered":"2024-09-22T03:23:26.291408+0000","last_clean":"2024-09-22T03:23:26.291408+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:23:26.291408+0000","last_undegraded":"2024-09-22T03:23:26.291408+0000","last_fullsized":"2024-09-22T03:23:26.291408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.006000+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:26.006000+0000","last_peered":"2024-09-22T03:23:26.006000+0000","last_clean":"2024-09-22T03:23:26.006000+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:26.006000+0000","last_undegraded":"2024-09-22T03:23:26.006000+0000","last_fullsized":"2024-09-22T03:23:26.006000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.372596+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:23:26.372596+0000","last_peered":"2024-09-22T03:23:26.372596+0000","last_clean":"2024-09-22T03:23:26.372596+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:23:26.372596+0000","last_undegraded":"2024-09-22T03:23:26.372596+0000","last_fullsized":"2024-09-22T03:23:26.372596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":30,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.038632+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:26.038632+0000","last_peered":"2024-09-22T03:23:26.038632+0000","last_clean":"2024-09-22T03:23:26.038632+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:26.038632+0000","last_undegraded":"2024-09-22T03:23:26.038632+0000","last_fullsized":"2024-09-22T03:23:26.038632+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"20'103","reported_seq":119,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.828611+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:23:26.828611+0000","last_peered":"2024-09-22T03:23:26.828611+0000","last_clean":"2024-09-22T03:23:26.828611+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:23:26.828611+0000","last_undegraded":"2024-09-22T03:23:26.828611+0000","last_fullsized":"2024-09-22T03:23:26.828611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.650883+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:23:26.650883+0000","last_peered":"2024-09-22T03:23:26.650883+0000","last_clean":"2024-09-22T03:23:26.650883+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:23:26.650883+0000","last_undegraded":"2024-09-22T03:23:26.650883+0000","last_fullsized":"2024-09-22T03:23:26.650883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.004959+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:26.004959+0000","last_peered":"2024-09-22T03:23:26.004959+0000","last_clean":"2024-09-22T03:23:26.004959+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:26.004959+0000","last_undegraded":"2024-09-22T03:23:26.004959+0000","last_fullsized":"2024-09-22T03:23:26.004959+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"20'144","reported_seq":160,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.705665+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:23:26.705665+0000","last_peered":"2024-09-22T03:23:26.705665+0000","last_clean":"2024-09-22T03:23:26.705665+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:23:26.705665+0000","last_undegraded":"2024-09-22T03:23:26.705665+0000","last_fullsized":"2024-09-22T03:23:26.705665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1187840,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":1160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.004151+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:26.004151+0000","last_peered":"2024-09-22T03:23:26.004151+0000","last_clean":"2024-09-22T03:23:26.004151+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:26.004151+0000","last_undegraded":"2024-09-22T03:23:26.004151+0000","last_fullsized":"2024-09-22T03:23:26.004151+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.488254+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:23:26.488254+0000","last_peered":"2024-09-22T03:23:26.488254+0000","last_clean":"2024-09-22T03:23:26.488254+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:23:26.488254+0000","last_undegraded":"2024-09-22T03:23:26.488254+0000","last_fullsized":"2024-09-22T03:23:26.488254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.003906+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:26.003906+0000","last_peered":"2024-09-22T03:23:26.003906+0000","last_clean":"2024-09-22T03:23:26.003906+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:26.003906+0000","last_undegraded":"2024-09-22T03:23:26.003906+0000","last_fullsized":"2024-09-22T03:23:26.003906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"20'160","reported_seq":176,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.872962+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:23:26.872962+0000","last_peered":"2024-09-22T03:23:26.872962+0000","last_clean":"2024-09-22T03:23:26.872962+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:23:26.872962+0000","last_undegraded":"2024-09-22T03:23:26.872962+0000","last_fullsized":"2024-09-22T03:23:26.872962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.008631+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:26.008631+0000","last_peered":"2024-09-22T03:23:26.008631+0000","last_clean":"2024-09-22T03:23:26.008631+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:26.008631+0000","last_undegraded":"2024-09-22T03:23:26.008631+0000","last_fullsized":"2024-09-22T03:23:26.008631+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":16334848,"num_objects":251,"num_object_clones":0,"num_object_copies":753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214103990272,"internally_reserved":0,"allocated":107503616,"data_stored":107503616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1982,"ondisk_log_size":1982,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214103990272,"internally_reserved":0,"allocated":107503616,"data_stored":107503616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":29772,"kb_used_data":29772,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52267956,"statfs":{"total":53552873472,"available":53522386944,"internally_reserved":0,"allocated":30486528,"data_stored":30486528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":24648,"kb_used_data":24648,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52273080,"statfs":{"total":53552873472,"available":53527633920,"internally_reserved":0,"allocated":25239552,"data_stored":25239552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":25828,"kb_used_data":25828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52271900,"statfs":{"total":53552873472,"available":53526425600,"internally_reserved":0,"allocated":26447872,"data_stored":26447872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23476,"kb_used_data":23476,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274252,"statfs":{"total":53552873472,"available":53528834048,"internally_reserved":0,"allocated":24039424,"data_stored":24039424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53528498176,"internally_reserved":0,"allocated":24375296,"data_stored":24375296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53526196224,"internally_reserved":0,"allocated":26677248,"data_stored":26677248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53527322624,"internally_reserved":0,"allocated":25550848,"data_stored":25550848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53528498176,"internally_reserved":0,"allocated":24375296,"data_stored":24375296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53526196224,"internally_reserved":0,"allocated":26677248,"data_stored":26677248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53527322624,"internally_reserved":0,"allocated":25550848,"data_stored":25550848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:29.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:29.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:29.529+0000 7f0a09f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:29.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:29.538+0000 7f0a09f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:29.800 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:29.801 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:29.821 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-22T03:23:29.004172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16925235,"num_objects":255,"num_object_clones":0,"num_object_copies":765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2133,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2082,"ondisk_log_size":2082,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":103724,"kb_used_data":103724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209087188,"statfs":{"total":214211493888,"available":214105280512,"internally_reserved":0,"allocated":106213376,"data_stored":106213376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7938048,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":969,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.064298"},"pg_stats":[{"pgid":"3.f","version":"20'143","reported_seq":159,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.400431+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:23:26.400431+0000","last_peered":"2024-09-22T03:23:26.400431+0000","last_clean":"2024-09-22T03:23:26.400431+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:23:26.400431+0000","last_undegraded":"2024-09-22T03:23:26.400431+0000","last_fullsized":"2024-09-22T03:23:26.400431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"20'135","reported_seq":151,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.738054+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:23:26.738054+0000","last_peered":"2024-09-22T03:23:26.738054+0000","last_clean":"2024-09-22T03:23:26.738054+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:23:26.738054+0000","last_undegraded":"2024-09-22T03:23:26.738054+0000","last_fullsized":"2024-09-22T03:23:26.738054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'111","reported_seq":127,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.439217+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:23:26.439217+0000","last_peered":"2024-09-22T03:23:26.439217+0000","last_clean":"2024-09-22T03:23:26.439217+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:23:26.439217+0000","last_undegraded":"2024-09-22T03:23:26.439217+0000","last_fullsized":"2024-09-22T03:23:26.439217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"19'104","reported_seq":119,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.037256+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:23:26.037256+0000","last_peered":"2024-09-22T03:23:26.037256+0000","last_clean":"2024-09-22T03:23:26.037256+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:23:26.037256+0000","last_undegraded":"2024-09-22T03:23:26.037256+0000","last_fullsized":"2024-09-22T03:23:26.037256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'119","reported_seq":135,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.523856+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:23:26.523856+0000","last_peered":"2024-09-22T03:23:26.523856+0000","last_clean":"2024-09-22T03:23:26.523856+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:23:26.523856+0000","last_undegraded":"2024-09-22T03:23:26.523856+0000","last_fullsized":"2024-09-22T03:23:26.523856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"19'96","reported_seq":111,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.035054+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:23:26.035054+0000","last_peered":"2024-09-22T03:23:26.035054+0000","last_clean":"2024-09-22T03:23:26.035054+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:23:26.035054+0000","last_undegraded":"2024-09-22T03:23:26.035054+0000","last_fullsized":"2024-09-22T03:23:26.035054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"20'167","reported_seq":183,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.792217+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:23:26.792217+0000","last_peered":"2024-09-22T03:23:26.792217+0000","last_clean":"2024-09-22T03:23:26.792217+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:23:26.792217+0000","last_undegraded":"2024-09-22T03:23:26.792217+0000","last_fullsized":"2024-09-22T03:23:26.792217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"19'112","reported_seq":127,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.005336+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:23:26.005336+0000","last_peered":"2024-09-22T03:23:26.005336+0000","last_clean":"2024-09-22T03:23:26.005336+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:23:26.005336+0000","last_undegraded":"2024-09-22T03:23:26.005336+0000","last_fullsized":"2024-09-22T03:23:26.005336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.001626+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:26.001626+0000","last_peered":"2024-09-22T03:23:26.001626+0000","last_clean":"2024-09-22T03:23:26.001626+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:26.001626+0000","last_undegraded":"2024-09-22T03:23:26.001626+0000","last_fullsized":"2024-09-22T03:23:26.001626+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":29,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.009152+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:26.009152+0000","last_peered":"2024-09-22T03:23:26.009152+0000","last_clean":"2024-09-22T03:23:26.009152+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:26.009152+0000","last_undegraded":"2024-09-22T03:23:26.009152+0000","last_fullsized":"2024-09-22T03:23:26.009152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"19'88","reported_seq":103,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.008412+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:23:26.008412+0000","last_peered":"2024-09-22T03:23:26.008412+0000","last_clean":"2024-09-22T03:23:26.008412+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:23:26.008412+0000","last_undegraded":"2024-09-22T03:23:26.008412+0000","last_fullsized":"2024-09-22T03:23:26.008412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.040503+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:26.040503+0000","last_peered":"2024-09-22T03:23:26.040503+0000","last_clean":"2024-09-22T03:23:26.040503+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:26.040503+0000","last_undegraded":"2024-09-22T03:23:26.040503+0000","last_fullsized":"2024-09-22T03:23:26.040503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"20'119","reported_seq":135,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.291408+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:23:26.291408+0000","last_peered":"2024-09-22T03:23:26.291408+0000","last_clean":"2024-09-22T03:23:26.291408+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:23:26.291408+0000","last_undegraded":"2024-09-22T03:23:26.291408+0000","last_fullsized":"2024-09-22T03:23:26.291408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.006000+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:26.006000+0000","last_peered":"2024-09-22T03:23:26.006000+0000","last_clean":"2024-09-22T03:23:26.006000+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:26.006000+0000","last_undegraded":"2024-09-22T03:23:26.006000+0000","last_fullsized":"2024-09-22T03:23:26.006000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.372596+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:23:26.372596+0000","last_peered":"2024-09-22T03:23:26.372596+0000","last_clean":"2024-09-22T03:23:26.372596+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:23:26.372596+0000","last_undegraded":"2024-09-22T03:23:26.372596+0000","last_fullsized":"2024-09-22T03:23:26.372596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":30,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.038632+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:26.038632+0000","last_peered":"2024-09-22T03:23:26.038632+0000","last_clean":"2024-09-22T03:23:26.038632+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:26.038632+0000","last_undegraded":"2024-09-22T03:23:26.038632+0000","last_fullsized":"2024-09-22T03:23:26.038632+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"20'103","reported_seq":119,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.828611+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:23:26.828611+0000","last_peered":"2024-09-22T03:23:26.828611+0000","last_clean":"2024-09-22T03:23:26.828611+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:23:26.828611+0000","last_undegraded":"2024-09-22T03:23:26.828611+0000","last_fullsized":"2024-09-22T03:23:26.828611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.650883+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:23:26.650883+0000","last_peered":"2024-09-22T03:23:26.650883+0000","last_clean":"2024-09-22T03:23:26.650883+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:23:26.650883+0000","last_undegraded":"2024-09-22T03:23:26.650883+0000","last_fullsized":"2024-09-22T03:23:26.650883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.004959+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:26.004959+0000","last_peered":"2024-09-22T03:23:26.004959+0000","last_clean":"2024-09-22T03:23:26.004959+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:26.004959+0000","last_undegraded":"2024-09-22T03:23:26.004959+0000","last_fullsized":"2024-09-22T03:23:26.004959+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"20'144","reported_seq":160,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.705665+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:23:26.705665+0000","last_peered":"2024-09-22T03:23:26.705665+0000","last_clean":"2024-09-22T03:23:26.705665+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:23:26.705665+0000","last_undegraded":"2024-09-22T03:23:26.705665+0000","last_fullsized":"2024-09-22T03:23:26.705665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1187840,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":1160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.004151+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:26.004151+0000","last_peered":"2024-09-22T03:23:26.004151+0000","last_clean":"2024-09-22T03:23:26.004151+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:26.004151+0000","last_undegraded":"2024-09-22T03:23:26.004151+0000","last_fullsized":"2024-09-22T03:23:26.004151+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.488254+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:23:26.488254+0000","last_peered":"2024-09-22T03:23:26.488254+0000","last_clean":"2024-09-22T03:23:26.488254+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:23:26.488254+0000","last_undegraded":"2024-09-22T03:23:26.488254+0000","last_fullsized":"2024-09-22T03:23:26.488254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.003906+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:26.003906+0000","last_peered":"2024-09-22T03:23:26.003906+0000","last_clean":"2024-09-22T03:23:26.003906+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:26.003906+0000","last_undegraded":"2024-09-22T03:23:26.003906+0000","last_fullsized":"2024-09-22T03:23:26.003906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"20'160","reported_seq":176,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.872962+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:23:26.872962+0000","last_peered":"2024-09-22T03:23:26.872962+0000","last_clean":"2024-09-22T03:23:26.872962+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:23:26.872962+0000","last_undegraded":"2024-09-22T03:23:26.872962+0000","last_fullsized":"2024-09-22T03:23:26.872962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.008631+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:26.008631+0000","last_peered":"2024-09-22T03:23:26.008631+0000","last_clean":"2024-09-22T03:23:26.008631+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:26.008631+0000","last_undegraded":"2024-09-22T03:23:26.008631+0000","last_fullsized":"2024-09-22T03:23:26.008631+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":16334848,"num_objects":251,"num_object_clones":0,"num_object_copies":753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214103990272,"internally_reserved":0,"allocated":107503616,"data_stored":107503616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1982,"ondisk_log_size":1982,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214103990272,"internally_reserved":0,"allocated":107503616,"data_stored":107503616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":29772,"kb_used_data":29772,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52267956,"statfs":{"total":53552873472,"available":53522386944,"internally_reserved":0,"allocated":30486528,"data_stored":30486528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":24648,"kb_used_data":24648,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52273080,"statfs":{"total":53552873472,"available":53527633920,"internally_reserved":0,"allocated":25239552,"data_stored":25239552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":25828,"kb_used_data":25828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52271900,"statfs":{"total":53552873472,"available":53526425600,"internally_reserved":0,"allocated":26447872,"data_stored":26447872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23476,"kb_used_data":23476,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274252,"statfs":{"total":53552873472,"available":53528834048,"internally_reserved":0,"allocated":24039424,"data_stored":24039424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53528498176,"internally_reserved":0,"allocated":24375296,"data_stored":24375296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53526196224,"internally_reserved":0,"allocated":26677248,"data_stored":26677248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53527322624,"internally_reserved":0,"allocated":25550848,"data_stored":25550848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53528498176,"internally_reserved":0,"allocated":24375296,"data_stored":24375296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53526196224,"internally_reserved":0,"allocated":26677248,"data_stored":26677248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53527322624,"internally_reserved":0,"allocated":25550848,"data_stored":25550848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:29.822 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:23:29.822 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:29.823 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:29.869 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:29.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:29.937+0000 7f448fe2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:29.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:29.946+0000 7f448fe2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:30.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:30.005+0000 7f24c7a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:30.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:30.014+0000 7f24c7a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:30.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 2621 2605 1.56538 1.46094 0.0692917 0.0796572 2024-09-22T03:23:30.211 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:30.211 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:30.229 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-22T03:23:29.941154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16925235,"num_objects":255,"num_object_clones":0,"num_object_copies":765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2133,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2082,"ondisk_log_size":2082,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":103724,"kb_used_data":103724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209087188,"statfs":{"total":214211493888,"available":214105280512,"internally_reserved":0,"allocated":106213376,"data_stored":106213376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7938048,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":969,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000753"},"pg_stats":[{"pgid":"3.f","version":"20'143","reported_seq":159,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.400431+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:23:26.400431+0000","last_peered":"2024-09-22T03:23:26.400431+0000","last_clean":"2024-09-22T03:23:26.400431+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:23:26.400431+0000","last_undegraded":"2024-09-22T03:23:26.400431+0000","last_fullsized":"2024-09-22T03:23:26.400431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"20'135","reported_seq":151,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.738054+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:23:26.738054+0000","last_peered":"2024-09-22T03:23:26.738054+0000","last_clean":"2024-09-22T03:23:26.738054+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:23:26.738054+0000","last_undegraded":"2024-09-22T03:23:26.738054+0000","last_fullsized":"2024-09-22T03:23:26.738054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'111","reported_seq":127,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.439217+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:23:26.439217+0000","last_peered":"2024-09-22T03:23:26.439217+0000","last_clean":"2024-09-22T03:23:26.439217+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:23:26.439217+0000","last_undegraded":"2024-09-22T03:23:26.439217+0000","last_fullsized":"2024-09-22T03:23:26.439217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"19'104","reported_seq":119,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.037256+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:23:26.037256+0000","last_peered":"2024-09-22T03:23:26.037256+0000","last_clean":"2024-09-22T03:23:26.037256+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:23:26.037256+0000","last_undegraded":"2024-09-22T03:23:26.037256+0000","last_fullsized":"2024-09-22T03:23:26.037256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'119","reported_seq":135,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.523856+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:23:26.523856+0000","last_peered":"2024-09-22T03:23:26.523856+0000","last_clean":"2024-09-22T03:23:26.523856+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:23:26.523856+0000","last_undegraded":"2024-09-22T03:23:26.523856+0000","last_fullsized":"2024-09-22T03:23:26.523856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"19'96","reported_seq":111,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.035054+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:23:26.035054+0000","last_peered":"2024-09-22T03:23:26.035054+0000","last_clean":"2024-09-22T03:23:26.035054+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:23:26.035054+0000","last_undegraded":"2024-09-22T03:23:26.035054+0000","last_fullsized":"2024-09-22T03:23:26.035054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"20'167","reported_seq":183,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.792217+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:23:26.792217+0000","last_peered":"2024-09-22T03:23:26.792217+0000","last_clean":"2024-09-22T03:23:26.792217+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:23:26.792217+0000","last_undegraded":"2024-09-22T03:23:26.792217+0000","last_fullsized":"2024-09-22T03:23:26.792217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"19'112","reported_seq":127,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.005336+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:23:26.005336+0000","last_peered":"2024-09-22T03:23:26.005336+0000","last_clean":"2024-09-22T03:23:26.005336+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:23:26.005336+0000","last_undegraded":"2024-09-22T03:23:26.005336+0000","last_fullsized":"2024-09-22T03:23:26.005336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.001626+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:26.001626+0000","last_peered":"2024-09-22T03:23:26.001626+0000","last_clean":"2024-09-22T03:23:26.001626+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:26.001626+0000","last_undegraded":"2024-09-22T03:23:26.001626+0000","last_fullsized":"2024-09-22T03:23:26.001626+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":29,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.009152+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:26.009152+0000","last_peered":"2024-09-22T03:23:26.009152+0000","last_clean":"2024-09-22T03:23:26.009152+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:26.009152+0000","last_undegraded":"2024-09-22T03:23:26.009152+0000","last_fullsized":"2024-09-22T03:23:26.009152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"19'88","reported_seq":103,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.008412+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:23:26.008412+0000","last_peered":"2024-09-22T03:23:26.008412+0000","last_clean":"2024-09-22T03:23:26.008412+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:23:26.008412+0000","last_undegraded":"2024-09-22T03:23:26.008412+0000","last_fullsized":"2024-09-22T03:23:26.008412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.040503+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:26.040503+0000","last_peered":"2024-09-22T03:23:26.040503+0000","last_clean":"2024-09-22T03:23:26.040503+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:26.040503+0000","last_undegraded":"2024-09-22T03:23:26.040503+0000","last_fullsized":"2024-09-22T03:23:26.040503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"20'119","reported_seq":135,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.291408+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:23:26.291408+0000","last_peered":"2024-09-22T03:23:26.291408+0000","last_clean":"2024-09-22T03:23:26.291408+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:23:26.291408+0000","last_undegraded":"2024-09-22T03:23:26.291408+0000","last_fullsized":"2024-09-22T03:23:26.291408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.006000+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:26.006000+0000","last_peered":"2024-09-22T03:23:26.006000+0000","last_clean":"2024-09-22T03:23:26.006000+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:26.006000+0000","last_undegraded":"2024-09-22T03:23:26.006000+0000","last_fullsized":"2024-09-22T03:23:26.006000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.372596+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:23:26.372596+0000","last_peered":"2024-09-22T03:23:26.372596+0000","last_clean":"2024-09-22T03:23:26.372596+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:23:26.372596+0000","last_undegraded":"2024-09-22T03:23:26.372596+0000","last_fullsized":"2024-09-22T03:23:26.372596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":30,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.038632+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:26.038632+0000","last_peered":"2024-09-22T03:23:26.038632+0000","last_clean":"2024-09-22T03:23:26.038632+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:26.038632+0000","last_undegraded":"2024-09-22T03:23:26.038632+0000","last_fullsized":"2024-09-22T03:23:26.038632+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"20'103","reported_seq":119,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.828611+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:23:26.828611+0000","last_peered":"2024-09-22T03:23:26.828611+0000","last_clean":"2024-09-22T03:23:26.828611+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:23:26.828611+0000","last_undegraded":"2024-09-22T03:23:26.828611+0000","last_fullsized":"2024-09-22T03:23:26.828611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.650883+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:23:26.650883+0000","last_peered":"2024-09-22T03:23:26.650883+0000","last_clean":"2024-09-22T03:23:26.650883+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:23:26.650883+0000","last_undegraded":"2024-09-22T03:23:26.650883+0000","last_fullsized":"2024-09-22T03:23:26.650883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.004959+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:26.004959+0000","last_peered":"2024-09-22T03:23:26.004959+0000","last_clean":"2024-09-22T03:23:26.004959+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:26.004959+0000","last_undegraded":"2024-09-22T03:23:26.004959+0000","last_fullsized":"2024-09-22T03:23:26.004959+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"20'144","reported_seq":160,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.705665+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:23:26.705665+0000","last_peered":"2024-09-22T03:23:26.705665+0000","last_clean":"2024-09-22T03:23:26.705665+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:23:26.705665+0000","last_undegraded":"2024-09-22T03:23:26.705665+0000","last_fullsized":"2024-09-22T03:23:26.705665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1187840,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":1160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.004151+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:26.004151+0000","last_peered":"2024-09-22T03:23:26.004151+0000","last_clean":"2024-09-22T03:23:26.004151+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:26.004151+0000","last_undegraded":"2024-09-22T03:23:26.004151+0000","last_fullsized":"2024-09-22T03:23:26.004151+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'127","reported_seq":143,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.488254+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:23:26.488254+0000","last_peered":"2024-09-22T03:23:26.488254+0000","last_clean":"2024-09-22T03:23:26.488254+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:23:26.488254+0000","last_undegraded":"2024-09-22T03:23:26.488254+0000","last_fullsized":"2024-09-22T03:23:26.488254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.003906+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:26.003906+0000","last_peered":"2024-09-22T03:23:26.003906+0000","last_clean":"2024-09-22T03:23:26.003906+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:26.003906+0000","last_undegraded":"2024-09-22T03:23:26.003906+0000","last_fullsized":"2024-09-22T03:23:26.003906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"20'160","reported_seq":176,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.872962+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:23:26.872962+0000","last_peered":"2024-09-22T03:23:26.872962+0000","last_clean":"2024-09-22T03:23:26.872962+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:23:26.872962+0000","last_undegraded":"2024-09-22T03:23:26.872962+0000","last_fullsized":"2024-09-22T03:23:26.872962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-22T03:23:26.008631+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:26.008631+0000","last_peered":"2024-09-22T03:23:26.008631+0000","last_clean":"2024-09-22T03:23:26.008631+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:26.008631+0000","last_undegraded":"2024-09-22T03:23:26.008631+0000","last_fullsized":"2024-09-22T03:23:26.008631+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":16334848,"num_objects":251,"num_object_clones":0,"num_object_copies":753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214103990272,"internally_reserved":0,"allocated":107503616,"data_stored":107503616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1982,"ondisk_log_size":1982,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214103990272,"internally_reserved":0,"allocated":107503616,"data_stored":107503616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":29772,"kb_used_data":29772,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52267956,"statfs":{"total":53552873472,"available":53522386944,"internally_reserved":0,"allocated":30486528,"data_stored":30486528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":24648,"kb_used_data":24648,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52273080,"statfs":{"total":53552873472,"available":53527633920,"internally_reserved":0,"allocated":25239552,"data_stored":25239552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":25828,"kb_used_data":25828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52271900,"statfs":{"total":53552873472,"available":53526425600,"internally_reserved":0,"allocated":26447872,"data_stored":26447872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705689,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23476,"kb_used_data":23476,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274252,"statfs":{"total":53552873472,"available":53528834048,"internally_reserved":0,"allocated":24039424,"data_stored":24039424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53528498176,"internally_reserved":0,"allocated":24375296,"data_stored":24375296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53526196224,"internally_reserved":0,"allocated":26677248,"data_stored":26677248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53527322624,"internally_reserved":0,"allocated":25550848,"data_stored":25550848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53528498176,"internally_reserved":0,"allocated":24375296,"data_stored":24375296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53526196224,"internally_reserved":0,"allocated":26677248,"data_stored":26677248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53527322624,"internally_reserved":0,"allocated":25550848,"data_stored":25550848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53521973248,"internally_reserved":0,"allocated":30900224,"data_stored":30900224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:31.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 2841 2825 1.57632 1.71875 0.0703629 0.0791323 2024-09-22T03:23:32.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 3034 3018 1.57174 1.50781 0.0358238 0.0792859 2024-09-22T03:23:32.135 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:32.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:32.273+0000 7f09fba6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:32.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:32.278+0000 7f09fba6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:33.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 3219 3203 1.56383 1.44531 0.0351861 0.0797602 2024-09-22T03:23:34.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 3425 3409 1.5665 1.60938 0.0561382 0.0795183 2024-09-22T03:23:34.453 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:34.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:34.563+0000 7f7eb1ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:34.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:34.569+0000 7f7eb1ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:35.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 3611 3595 1.56019 1.45312 0.0851663 0.0797894 2024-09-22T03:23:35.230 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:35.231 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:35.231 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:35.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:35.359+0000 7f517442c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:35.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:35.366+0000 7f517442c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:35.662 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:35.679 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-22T03:23:33.941921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25256499,"num_objects":381,"num_object_clones":0,"num_object_copies":1143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3150,"num_write_kb":25984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3095,"ondisk_log_size":3095,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":152780,"kb_used_data":152780,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209038132,"statfs":{"total":214211493888,"available":214055047168,"internally_reserved":0,"allocated":156446720,"data_stored":156446720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8331264,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1017,"num_write_kb":8136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.950657"},"pg_stats":[{"pgid":"3.f","version":"23'191","reported_seq":216,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.168630+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:23:31.168630+0000","last_peered":"2024-09-22T03:23:31.168630+0000","last_clean":"2024-09-22T03:23:31.168630+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:23:31.168630+0000","last_undegraded":"2024-09-22T03:23:31.168630+0000","last_fullsized":"2024-09-22T03:23:31.168630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'207","reported_seq":232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.582861+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:23:31.582861+0000","last_peered":"2024-09-22T03:23:31.582861+0000","last_clean":"2024-09-22T03:23:31.582861+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:23:31.582861+0000","last_undegraded":"2024-09-22T03:23:31.582861+0000","last_fullsized":"2024-09-22T03:23:31.582861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'183","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.953210+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:23:29.953210+0000","last_peered":"2024-09-22T03:23:29.953210+0000","last_clean":"2024-09-22T03:23:29.953210+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:23:29.953210+0000","last_undegraded":"2024-09-22T03:23:29.953210+0000","last_fullsized":"2024-09-22T03:23:29.953210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'151","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.542057+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:23:31.542057+0000","last_peered":"2024-09-22T03:23:31.542057+0000","last_clean":"2024-09-22T03:23:31.542057+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:23:31.542057+0000","last_undegraded":"2024-09-22T03:23:31.542057+0000","last_fullsized":"2024-09-22T03:23:31.542057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'159","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:30.784098+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:23:30.784098+0000","last_peered":"2024-09-22T03:23:30.784098+0000","last_clean":"2024-09-22T03:23:30.784098+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:23:30.784098+0000","last_undegraded":"2024-09-22T03:23:30.784098+0000","last_fullsized":"2024-09-22T03:23:30.784098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'199","reported_seq":224,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.448788+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:23:31.448788+0000","last_peered":"2024-09-22T03:23:31.448788+0000","last_clean":"2024-09-22T03:23:31.448788+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:23:31.448788+0000","last_undegraded":"2024-09-22T03:23:31.448788+0000","last_fullsized":"2024-09-22T03:23:31.448788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"23'207","reported_seq":232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.479070+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:23:31.479070+0000","last_peered":"2024-09-22T03:23:31.479070+0000","last_clean":"2024-09-22T03:23:31.479070+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:23:31.479070+0000","last_undegraded":"2024-09-22T03:23:31.479070+0000","last_fullsized":"2024-09-22T03:23:31.479070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'199","reported_seq":224,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.773788+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:23:31.773788+0000","last_peered":"2024-09-22T03:23:31.773788+0000","last_clean":"2024-09-22T03:23:31.773788+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:23:31.773788+0000","last_undegraded":"2024-09-22T03:23:31.773788+0000","last_fullsized":"2024-09-22T03:23:31.773788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.026474+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:29.026474+0000","last_peered":"2024-09-22T03:23:29.026474+0000","last_clean":"2024-09-22T03:23:29.026474+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:29.026474+0000","last_undegraded":"2024-09-22T03:23:29.026474+0000","last_fullsized":"2024-09-22T03:23:29.026474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.055016+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:29.055016+0000","last_peered":"2024-09-22T03:23:29.055016+0000","last_clean":"2024-09-22T03:23:29.055016+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:29.055016+0000","last_undegraded":"2024-09-22T03:23:29.055016+0000","last_fullsized":"2024-09-22T03:23:29.055016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'167","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:30.970283+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:23:30.970283+0000","last_peered":"2024-09-22T03:23:30.970283+0000","last_clean":"2024-09-22T03:23:30.970283+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:23:30.970283+0000","last_undegraded":"2024-09-22T03:23:30.970283+0000","last_fullsized":"2024-09-22T03:23:30.970283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.047802+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:29.047802+0000","last_peered":"2024-09-22T03:23:29.047802+0000","last_clean":"2024-09-22T03:23:29.047802+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:29.047802+0000","last_undegraded":"2024-09-22T03:23:29.047802+0000","last_fullsized":"2024-09-22T03:23:29.047802+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'183","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:30.584990+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:23:30.584990+0000","last_peered":"2024-09-22T03:23:30.584990+0000","last_clean":"2024-09-22T03:23:30.584990+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:23:30.584990+0000","last_undegraded":"2024-09-22T03:23:30.584990+0000","last_fullsized":"2024-09-22T03:23:30.584990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.051712+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.051712+0000","last_peered":"2024-09-22T03:23:29.051712+0000","last_clean":"2024-09-22T03:23:29.051712+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:29.051712+0000","last_undegraded":"2024-09-22T03:23:29.051712+0000","last_fullsized":"2024-09-22T03:23:29.051712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'207","reported_seq":232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.542874+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:23:31.542874+0000","last_peered":"2024-09-22T03:23:31.542874+0000","last_clean":"2024-09-22T03:23:31.542874+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:23:31.542874+0000","last_undegraded":"2024-09-22T03:23:31.542874+0000","last_fullsized":"2024-09-22T03:23:31.542874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.043472+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.043472+0000","last_peered":"2024-09-22T03:23:29.043472+0000","last_clean":"2024-09-22T03:23:29.043472+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:29.043472+0000","last_undegraded":"2024-09-22T03:23:29.043472+0000","last_fullsized":"2024-09-22T03:23:29.043472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'127","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.646585+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:23:29.646585+0000","last_peered":"2024-09-22T03:23:29.646585+0000","last_clean":"2024-09-22T03:23:29.646585+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:23:29.646585+0000","last_undegraded":"2024-09-22T03:23:29.646585+0000","last_fullsized":"2024-09-22T03:23:29.646585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'181","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.942644+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:23:31.942644+0000","last_peered":"2024-09-22T03:23:31.942644+0000","last_clean":"2024-09-22T03:23:31.942644+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:23:31.942644+0000","last_undegraded":"2024-09-22T03:23:31.942644+0000","last_fullsized":"2024-09-22T03:23:31.942644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.039397+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.039397+0000","last_peered":"2024-09-22T03:23:29.039397+0000","last_clean":"2024-09-22T03:23:29.039397+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:29.039397+0000","last_undegraded":"2024-09-22T03:23:29.039397+0000","last_fullsized":"2024-09-22T03:23:29.039397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'207","reported_seq":232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.897709+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:23:31.897709+0000","last_peered":"2024-09-22T03:23:31.897709+0000","last_clean":"2024-09-22T03:23:31.897709+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:23:31.897709+0000","last_undegraded":"2024-09-22T03:23:31.897709+0000","last_fullsized":"2024-09-22T03:23:31.897709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.028966+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.028966+0000","last_peered":"2024-09-22T03:23:29.028966+0000","last_clean":"2024-09-22T03:23:29.028966+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:29.028966+0000","last_undegraded":"2024-09-22T03:23:29.028966+0000","last_fullsized":"2024-09-22T03:23:29.028966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'188","reported_seq":213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.943839+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:23:31.943839+0000","last_peered":"2024-09-22T03:23:31.943839+0000","last_clean":"2024-09-22T03:23:31.943839+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:23:31.943839+0000","last_undegraded":"2024-09-22T03:23:31.943839+0000","last_fullsized":"2024-09-22T03:23:31.943839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548288,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.036076+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:29.036076+0000","last_peered":"2024-09-22T03:23:29.036076+0000","last_clean":"2024-09-22T03:23:29.036076+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:29.036076+0000","last_undegraded":"2024-09-22T03:23:29.036076+0000","last_fullsized":"2024-09-22T03:23:29.036076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'239","reported_seq":264,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:31.726887+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:23:31.726887+0000","last_peered":"2024-09-22T03:23:31.726887+0000","last_clean":"2024-09-22T03:23:31.726887+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:23:31.726887+0000","last_undegraded":"2024-09-22T03:23:31.726887+0000","last_fullsized":"2024-09-22T03:23:31.726887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.058166+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:29.058166+0000","last_peered":"2024-09-22T03:23:29.058166+0000","last_clean":"2024-09-22T03:23:29.058166+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:29.058166+0000","last_undegraded":"2024-09-22T03:23:29.058166+0000","last_fullsized":"2024-09-22T03:23:29.058166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":24666112,"num_objects":377,"num_object_clones":0,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3011,"num_write_kb":24088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214053679104,"internally_reserved":0,"allocated":157814784,"data_stored":157814784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2995,"ondisk_log_size":2995,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214053695488,"internally_reserved":0,"allocated":157798400,"data_stored":157798400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53507670016,"internally_reserved":0,"allocated":45203456,"data_stored":45203456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":43888,"kb_used_data":43888,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52253840,"statfs":{"total":53552873472,"available":53507932160,"internally_reserved":0,"allocated":44941312,"data_stored":44941312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":35832,"kb_used_data":35832,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52261896,"statfs":{"total":53552873472,"available":53516181504,"internally_reserved":0,"allocated":36691968,"data_stored":36691968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":39136,"kb_used_data":39136,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52258592,"statfs":{"total":53552873472,"available":53512798208,"internally_reserved":0,"allocated":40075264,"data_stored":40075264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":33924,"kb_used_data":33924,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52263804,"statfs":{"total":53552873472,"available":53518135296,"internally_reserved":0,"allocated":34738176,"data_stored":34738176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53507670016,"internally_reserved":0,"allocated":45203456,"data_stored":45203456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53517930496,"internally_reserved":0,"allocated":34942976,"data_stored":34942976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53512294400,"internally_reserved":0,"allocated":40579072,"data_stored":40579072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53515800576,"internally_reserved":0,"allocated":37072896,"data_stored":37072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53507670016,"internally_reserved":0,"allocated":45203456,"data_stored":45203456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53517930496,"internally_reserved":0,"allocated":34942976,"data_stored":34942976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53512294400,"internally_reserved":0,"allocated":40579072,"data_stored":40579072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53515792384,"internally_reserved":0,"allocated":37081088,"data_stored":37081088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53507661824,"internally_reserved":0,"allocated":45211648,"data_stored":45211648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:36.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 3784 3768 1.54921 1.35156 0.15336 0.0804774 2024-09-22T03:23:36.701 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:36.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:36.844+0000 7f1ecc658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:36.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:36.850+0000 7f1ecc658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:37.047 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:23:37.047303+0000 min lat: 0.0101625 max lat: 0.309215 avg lat: 0.0804041 2024-09-22T03:23:37.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:23:37.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 3974 3958 1.54596 1.48438 0.142915 0.0804041 2024-09-22T03:23:38.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 4170 4154 1.54525 1.53125 0.0942175 0.0807234 2024-09-22T03:23:38.948 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:39.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 4362 4346 1.54319 1.5 0.0914264 0.0808036 2024-09-22T03:23:39.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:39.065+0000 7f2b12f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:39.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:39.070+0000 7f2b12f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:40.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 4558 4542 1.54266 1.53125 0.0723903 0.0808759 2024-09-22T03:23:40.681 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:40.682 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:40.682 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:40.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:40.791+0000 7f771a597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:40.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:40.797+0000 7f771a597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:40.993 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:40.994 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:41.015 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-22T03:23:39.943242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32989747,"num_objects":500,"num_object_clones":0,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4094,"num_write_kb":33536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4039,"ondisk_log_size":4039,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":200612,"kb_used_data":200612,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208990300,"statfs":{"total":214211493888,"available":214006067200,"internally_reserved":0,"allocated":205426688,"data_stored":205426688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16064512,"num_objects":245,"num_object_clones":0,"num_object_copies":735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1961,"num_write_kb":15688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.939070"},"pg_stats":[{"pgid":"3.f","version":"23'239","reported_seq":264,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.711347+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:23:36.711347+0000","last_peered":"2024-09-22T03:23:36.711347+0000","last_clean":"2024-09-22T03:23:36.711347+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:23:36.711347+0000","last_undegraded":"2024-09-22T03:23:36.711347+0000","last_fullsized":"2024-09-22T03:23:36.711347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'289","reported_seq":314,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.881722+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:23:36.881722+0000","last_peered":"2024-09-22T03:23:36.881722+0000","last_clean":"2024-09-22T03:23:36.881722+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:23:36.881722+0000","last_undegraded":"2024-09-22T03:23:36.881722+0000","last_fullsized":"2024-09-22T03:23:36.881722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2375680,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'239","reported_seq":264,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.140948+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:23:36.140948+0000","last_peered":"2024-09-22T03:23:36.140948+0000","last_clean":"2024-09-22T03:23:36.140948+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:23:36.140948+0000","last_undegraded":"2024-09-22T03:23:36.140948+0000","last_fullsized":"2024-09-22T03:23:36.140948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'223","reported_seq":248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.560248+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:23:36.560248+0000","last_peered":"2024-09-22T03:23:36.560248+0000","last_clean":"2024-09-22T03:23:36.560248+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:23:36.560248+0000","last_undegraded":"2024-09-22T03:23:36.560248+0000","last_fullsized":"2024-09-22T03:23:36.560248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'223","reported_seq":248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.336611+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:23:36.336611+0000","last_peered":"2024-09-22T03:23:36.336611+0000","last_clean":"2024-09-22T03:23:36.336611+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:23:36.336611+0000","last_undegraded":"2024-09-22T03:23:36.336611+0000","last_fullsized":"2024-09-22T03:23:36.336611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'279","reported_seq":304,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.652357+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:23:36.652357+0000","last_peered":"2024-09-22T03:23:36.652357+0000","last_clean":"2024-09-22T03:23:36.652357+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:23:36.652357+0000","last_undegraded":"2024-09-22T03:23:36.652357+0000","last_fullsized":"2024-09-22T03:23:36.652357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"23'279","reported_seq":304,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.376747+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:23:36.376747+0000","last_peered":"2024-09-22T03:23:36.376747+0000","last_clean":"2024-09-22T03:23:36.376747+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:23:36.376747+0000","last_undegraded":"2024-09-22T03:23:36.376747+0000","last_fullsized":"2024-09-22T03:23:36.376747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'239","reported_seq":264,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.754451+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:23:36.754451+0000","last_peered":"2024-09-22T03:23:36.754451+0000","last_clean":"2024-09-22T03:23:36.754451+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:23:36.754451+0000","last_undegraded":"2024-09-22T03:23:36.754451+0000","last_fullsized":"2024-09-22T03:23:36.754451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.026474+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:29.026474+0000","last_peered":"2024-09-22T03:23:29.026474+0000","last_clean":"2024-09-22T03:23:29.026474+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:29.026474+0000","last_undegraded":"2024-09-22T03:23:29.026474+0000","last_fullsized":"2024-09-22T03:23:29.026474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.055016+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:29.055016+0000","last_peered":"2024-09-22T03:23:29.055016+0000","last_clean":"2024-09-22T03:23:29.055016+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:29.055016+0000","last_undegraded":"2024-09-22T03:23:29.055016+0000","last_fullsized":"2024-09-22T03:23:29.055016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'191","reported_seq":216,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:35.751223+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:23:35.751223+0000","last_peered":"2024-09-22T03:23:35.751223+0000","last_clean":"2024-09-22T03:23:35.751223+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:23:35.751223+0000","last_undegraded":"2024-09-22T03:23:35.751223+0000","last_fullsized":"2024-09-22T03:23:35.751223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.047802+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:29.047802+0000","last_peered":"2024-09-22T03:23:29.047802+0000","last_clean":"2024-09-22T03:23:29.047802+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:29.047802+0000","last_undegraded":"2024-09-22T03:23:29.047802+0000","last_fullsized":"2024-09-22T03:23:29.047802+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'223","reported_seq":248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:35.541221+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:23:35.541221+0000","last_peered":"2024-09-22T03:23:35.541221+0000","last_clean":"2024-09-22T03:23:35.541221+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:23:35.541221+0000","last_undegraded":"2024-09-22T03:23:35.541221+0000","last_fullsized":"2024-09-22T03:23:35.541221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.051712+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.051712+0000","last_peered":"2024-09-22T03:23:29.051712+0000","last_clean":"2024-09-22T03:23:29.051712+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:29.051712+0000","last_undegraded":"2024-09-22T03:23:29.051712+0000","last_fullsized":"2024-09-22T03:23:29.051712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'287","reported_seq":312,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.109804+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:23:36.109804+0000","last_peered":"2024-09-22T03:23:36.109804+0000","last_clean":"2024-09-22T03:23:36.109804+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:23:36.109804+0000","last_undegraded":"2024-09-22T03:23:36.109804+0000","last_fullsized":"2024-09-22T03:23:36.109804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.043472+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.043472+0000","last_peered":"2024-09-22T03:23:29.043472+0000","last_clean":"2024-09-22T03:23:29.043472+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:29.043472+0000","last_undegraded":"2024-09-22T03:23:29.043472+0000","last_fullsized":"2024-09-22T03:23:29.043472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'191","reported_seq":216,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.784754+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:23:36.784754+0000","last_peered":"2024-09-22T03:23:36.784754+0000","last_clean":"2024-09-22T03:23:36.784754+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:23:36.784754+0000","last_undegraded":"2024-09-22T03:23:36.784754+0000","last_fullsized":"2024-09-22T03:23:36.784754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'215","reported_seq":240,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.489304+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:23:36.489304+0000","last_peered":"2024-09-22T03:23:36.489304+0000","last_clean":"2024-09-22T03:23:36.489304+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:23:36.489304+0000","last_undegraded":"2024-09-22T03:23:36.489304+0000","last_fullsized":"2024-09-22T03:23:36.489304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.039397+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.039397+0000","last_peered":"2024-09-22T03:23:29.039397+0000","last_clean":"2024-09-22T03:23:29.039397+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:29.039397+0000","last_undegraded":"2024-09-22T03:23:29.039397+0000","last_fullsized":"2024-09-22T03:23:29.039397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'280","reported_seq":305,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.615897+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:23:36.615897+0000","last_peered":"2024-09-22T03:23:36.615897+0000","last_clean":"2024-09-22T03:23:36.615897+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:23:36.615897+0000","last_undegraded":"2024-09-22T03:23:36.615897+0000","last_fullsized":"2024-09-22T03:23:36.615897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2301952,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":281,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.028966+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.028966+0000","last_peered":"2024-09-22T03:23:29.028966+0000","last_clean":"2024-09-22T03:23:29.028966+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:29.028966+0000","last_undegraded":"2024-09-22T03:23:29.028966+0000","last_fullsized":"2024-09-22T03:23:29.028966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'239","reported_seq":264,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.861074+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:23:36.861074+0000","last_peered":"2024-09-22T03:23:36.861074+0000","last_clean":"2024-09-22T03:23:36.861074+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:23:36.861074+0000","last_undegraded":"2024-09-22T03:23:36.861074+0000","last_fullsized":"2024-09-22T03:23:36.861074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.036076+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:29.036076+0000","last_peered":"2024-09-22T03:23:29.036076+0000","last_clean":"2024-09-22T03:23:29.036076+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:29.036076+0000","last_undegraded":"2024-09-22T03:23:29.036076+0000","last_fullsized":"2024-09-22T03:23:29.036076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'303","reported_seq":328,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:36.218662+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:23:36.218662+0000","last_peered":"2024-09-22T03:23:36.218662+0000","last_clean":"2024-09-22T03:23:36.218662+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:23:36.218662+0000","last_undegraded":"2024-09-22T03:23:36.218662+0000","last_fullsized":"2024-09-22T03:23:36.218662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.058166+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:29.058166+0000","last_peered":"2024-09-22T03:23:29.058166+0000","last_clean":"2024-09-22T03:23:29.058166+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:29.058166+0000","last_undegraded":"2024-09-22T03:23:29.058166+0000","last_fullsized":"2024-09-22T03:23:29.058166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":32399360,"num_objects":496,"num_object_clones":0,"num_object_copies":1488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3955,"num_write_kb":31640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214004436992,"internally_reserved":0,"allocated":207056896,"data_stored":207056896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3939,"ondisk_log_size":3939,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214004436992,"internally_reserved":0,"allocated":207056896,"data_stored":207056896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53493895168,"internally_reserved":0,"allocated":58978304,"data_stored":58978304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":57064,"kb_used_data":57064,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52240664,"statfs":{"total":53552873472,"available":53494439936,"internally_reserved":0,"allocated":58433536,"data_stored":58433536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":48112,"kb_used_data":48112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52249616,"statfs":{"total":53552873472,"available":53503606784,"internally_reserved":0,"allocated":49266688,"data_stored":49266688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":50704,"kb_used_data":50704,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52247024,"statfs":{"total":53552873472,"available":53500952576,"internally_reserved":0,"allocated":51920896,"data_stored":51920896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":44732,"kb_used_data":44732,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52252996,"statfs":{"total":53552873472,"available":53507067904,"internally_reserved":0,"allocated":45805568,"data_stored":45805568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53493895168,"internally_reserved":0,"allocated":58978304,"data_stored":58978304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53506756608,"internally_reserved":0,"allocated":46116864,"data_stored":46116864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53500522496,"internally_reserved":0,"allocated":52350976,"data_stored":52350976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53503262720,"internally_reserved":0,"allocated":49610752,"data_stored":49610752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53493895168,"internally_reserved":0,"allocated":58978304,"data_stored":58978304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53506756608,"internally_reserved":0,"allocated":46116864,"data_stored":46116864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53500522496,"internally_reserved":0,"allocated":52350976,"data_stored":52350976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53503262720,"internally_reserved":0,"allocated":49610752,"data_stored":49610752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53493895168,"internally_reserved":0,"allocated":58978304,"data_stored":58978304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:41.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 4717 4701 1.53014 1.24219 0.0307118 0.0812938 2024-09-22T03:23:41.236 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:41.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:41.382+0000 7fe90d2b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:41.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:41.388+0000 7fe90d2b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:42.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 4905 4889 1.52768 1.46875 0.0747334 0.0814371 2024-09-22T03:23:43.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 5095 5079 1.52601 1.48438 0.106536 0.0817866 2024-09-22T03:23:43.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:43.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:43.658+0000 7fd3454fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:43.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:43.664+0000 7fd3454fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:44.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 5296 5280 1.52764 1.57031 0.0637215 0.0816351 2024-09-22T03:23:45.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 5483 5467 1.52526 1.46094 0.0736672 0.0818243 2024-09-22T03:23:45.906 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:46.016 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:46.017 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:46.017 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:46.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:46.032+0000 7f3b49818640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:46.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:46.038+0000 7f3b49818640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:46.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 5637 5621 1.51415 1.20312 0.105788 0.0824308 2024-09-22T03:23:46.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:46.165+0000 7f8b6b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:46.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:46.175+0000 7f8b6b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:46.435 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:46.435 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:46.456 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-22T03:23:45.944254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40657459,"num_objects":616,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5030,"num_write_kb":41024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4975,"ondisk_log_size":4975,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":247844,"kb_used_data":247844,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208943068,"statfs":{"total":214211493888,"available":213957701632,"internally_reserved":0,"allocated":253792256,"data_stored":253792256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15400960,"num_objects":235,"num_object_clones":0,"num_object_copies":705,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1880,"num_write_kb":15040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002333"},"pg_stats":[{"pgid":"3.f","version":"23'303","reported_seq":328,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.698133+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:23:41.698133+0000","last_peered":"2024-09-22T03:23:41.698133+0000","last_clean":"2024-09-22T03:23:41.698133+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:23:41.698133+0000","last_undegraded":"2024-09-22T03:23:41.698133+0000","last_fullsized":"2024-09-22T03:23:41.698133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'375","reported_seq":400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.854615+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:23:41.854615+0000","last_peered":"2024-09-22T03:23:41.854615+0000","last_clean":"2024-09-22T03:23:41.854615+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:23:41.854615+0000","last_undegraded":"2024-09-22T03:23:41.854615+0000","last_fullsized":"2024-09-22T03:23:41.854615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'295","reported_seq":320,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.220124+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:23:41.220124+0000","last_peered":"2024-09-22T03:23:41.220124+0000","last_clean":"2024-09-22T03:23:41.220124+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:23:41.220124+0000","last_undegraded":"2024-09-22T03:23:41.220124+0000","last_fullsized":"2024-09-22T03:23:41.220124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'271","reported_seq":296,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.347354+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:23:41.347354+0000","last_peered":"2024-09-22T03:23:41.347354+0000","last_clean":"2024-09-22T03:23:41.347354+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:23:41.347354+0000","last_undegraded":"2024-09-22T03:23:41.347354+0000","last_fullsized":"2024-09-22T03:23:41.347354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'287","reported_seq":312,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.059885+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:23:41.059885+0000","last_peered":"2024-09-22T03:23:41.059885+0000","last_clean":"2024-09-22T03:23:41.059885+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:23:41.059885+0000","last_undegraded":"2024-09-22T03:23:41.059885+0000","last_fullsized":"2024-09-22T03:23:41.059885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'311","reported_seq":336,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:40.538912+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:23:40.538912+0000","last_peered":"2024-09-22T03:23:40.538912+0000","last_clean":"2024-09-22T03:23:40.538912+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:23:40.538912+0000","last_undegraded":"2024-09-22T03:23:40.538912+0000","last_fullsized":"2024-09-22T03:23:40.538912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"23'343","reported_seq":368,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.140745+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:23:41.140745+0000","last_peered":"2024-09-22T03:23:41.140745+0000","last_clean":"2024-09-22T03:23:41.140745+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:23:41.140745+0000","last_undegraded":"2024-09-22T03:23:41.140745+0000","last_fullsized":"2024-09-22T03:23:41.140745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'263","reported_seq":288,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:38.204606+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:23:38.204606+0000","last_peered":"2024-09-22T03:23:38.204606+0000","last_clean":"2024-09-22T03:23:38.204606+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:23:38.204606+0000","last_undegraded":"2024-09-22T03:23:38.204606+0000","last_fullsized":"2024-09-22T03:23:38.204606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.026474+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:29.026474+0000","last_peered":"2024-09-22T03:23:29.026474+0000","last_clean":"2024-09-22T03:23:29.026474+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:29.026474+0000","last_undegraded":"2024-09-22T03:23:29.026474+0000","last_fullsized":"2024-09-22T03:23:29.026474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.055016+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:29.055016+0000","last_peered":"2024-09-22T03:23:29.055016+0000","last_clean":"2024-09-22T03:23:29.055016+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:29.055016+0000","last_undegraded":"2024-09-22T03:23:29.055016+0000","last_fullsized":"2024-09-22T03:23:29.055016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'247","reported_seq":272,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.481164+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:23:41.481164+0000","last_peered":"2024-09-22T03:23:41.481164+0000","last_clean":"2024-09-22T03:23:41.481164+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:23:41.481164+0000","last_undegraded":"2024-09-22T03:23:41.481164+0000","last_fullsized":"2024-09-22T03:23:41.481164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.047802+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:29.047802+0000","last_peered":"2024-09-22T03:23:29.047802+0000","last_clean":"2024-09-22T03:23:29.047802+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:29.047802+0000","last_undegraded":"2024-09-22T03:23:29.047802+0000","last_fullsized":"2024-09-22T03:23:29.047802+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'279","reported_seq":304,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:39.965907+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:23:39.965907+0000","last_peered":"2024-09-22T03:23:39.965907+0000","last_clean":"2024-09-22T03:23:39.965907+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:23:39.965907+0000","last_undegraded":"2024-09-22T03:23:39.965907+0000","last_fullsized":"2024-09-22T03:23:39.965907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.051712+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.051712+0000","last_peered":"2024-09-22T03:23:29.051712+0000","last_clean":"2024-09-22T03:23:29.051712+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:29.051712+0000","last_undegraded":"2024-09-22T03:23:29.051712+0000","last_fullsized":"2024-09-22T03:23:29.051712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'319","reported_seq":344,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.614322+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:23:41.614322+0000","last_peered":"2024-09-22T03:23:41.614322+0000","last_clean":"2024-09-22T03:23:41.614322+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:23:41.614322+0000","last_undegraded":"2024-09-22T03:23:41.614322+0000","last_fullsized":"2024-09-22T03:23:41.614322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.043472+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.043472+0000","last_peered":"2024-09-22T03:23:29.043472+0000","last_clean":"2024-09-22T03:23:29.043472+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:29.043472+0000","last_undegraded":"2024-09-22T03:23:29.043472+0000","last_fullsized":"2024-09-22T03:23:29.043472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'223","reported_seq":248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.164715+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:23:41.164715+0000","last_peered":"2024-09-22T03:23:41.164715+0000","last_clean":"2024-09-22T03:23:41.164715+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:23:41.164715+0000","last_undegraded":"2024-09-22T03:23:41.164715+0000","last_fullsized":"2024-09-22T03:23:41.164715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'319","reported_seq":344,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.809762+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:23:41.809762+0000","last_peered":"2024-09-22T03:23:41.809762+0000","last_clean":"2024-09-22T03:23:41.809762+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:23:41.809762+0000","last_undegraded":"2024-09-22T03:23:41.809762+0000","last_fullsized":"2024-09-22T03:23:41.809762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.039397+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.039397+0000","last_peered":"2024-09-22T03:23:29.039397+0000","last_clean":"2024-09-22T03:23:29.039397+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:29.039397+0000","last_undegraded":"2024-09-22T03:23:29.039397+0000","last_fullsized":"2024-09-22T03:23:29.039397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'394","reported_seq":419,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.920153+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:23:41.920153+0000","last_peered":"2024-09-22T03:23:41.920153+0000","last_clean":"2024-09-22T03:23:41.920153+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:23:41.920153+0000","last_undegraded":"2024-09-22T03:23:41.920153+0000","last_fullsized":"2024-09-22T03:23:41.920153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3235840,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":395,"num_write_kb":3160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.028966+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.028966+0000","last_peered":"2024-09-22T03:23:29.028966+0000","last_clean":"2024-09-22T03:23:29.028966+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:29.028966+0000","last_undegraded":"2024-09-22T03:23:29.028966+0000","last_fullsized":"2024-09-22T03:23:29.028966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'271","reported_seq":296,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:40.570904+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:23:40.570904+0000","last_peered":"2024-09-22T03:23:40.570904+0000","last_clean":"2024-09-22T03:23:40.570904+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:23:40.570904+0000","last_undegraded":"2024-09-22T03:23:40.570904+0000","last_fullsized":"2024-09-22T03:23:40.570904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.036076+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:29.036076+0000","last_peered":"2024-09-22T03:23:29.036076+0000","last_clean":"2024-09-22T03:23:29.036076+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:29.036076+0000","last_undegraded":"2024-09-22T03:23:29.036076+0000","last_fullsized":"2024-09-22T03:23:29.036076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'375","reported_seq":400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:41.886965+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:23:41.886965+0000","last_peered":"2024-09-22T03:23:41.886965+0000","last_clean":"2024-09-22T03:23:41.886965+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:23:41.886965+0000","last_undegraded":"2024-09-22T03:23:41.886965+0000","last_fullsized":"2024-09-22T03:23:41.886965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.058166+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:29.058166+0000","last_peered":"2024-09-22T03:23:29.058166+0000","last_clean":"2024-09-22T03:23:29.058166+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:29.058166+0000","last_undegraded":"2024-09-22T03:23:29.058166+0000","last_fullsized":"2024-09-22T03:23:29.058166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":40067072,"num_objects":612,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4891,"num_write_kb":39128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213956816896,"internally_reserved":0,"allocated":254676992,"data_stored":254676992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4875,"ondisk_log_size":4875,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213956816896,"internally_reserved":0,"allocated":254676992,"data_stored":254676992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53479510016,"internally_reserved":0,"allocated":73363456,"data_stored":73363456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":71260,"kb_used_data":71260,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52226468,"statfs":{"total":53552873472,"available":53479903232,"internally_reserved":0,"allocated":72970240,"data_stored":72970240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":59292,"kb_used_data":59292,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52238436,"statfs":{"total":53552873472,"available":53492158464,"internally_reserved":0,"allocated":60715008,"data_stored":60715008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705704,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":62084,"kb_used_data":62084,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52235644,"statfs":{"total":53552873472,"available":53489299456,"internally_reserved":0,"allocated":63574016,"data_stored":63574016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":55208,"kb_used_data":55208,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52242520,"statfs":{"total":53552873472,"available":53496340480,"internally_reserved":0,"allocated":56532992,"data_stored":56532992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53479510016,"internally_reserved":0,"allocated":73363456,"data_stored":73363456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53496250368,"internally_reserved":0,"allocated":56623104,"data_stored":56623104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53489053696,"internally_reserved":0,"allocated":63819776,"data_stored":63819776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53492002816,"internally_reserved":0,"allocated":60870656,"data_stored":60870656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53479510016,"internally_reserved":0,"allocated":73363456,"data_stored":73363456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53496250368,"internally_reserved":0,"allocated":56623104,"data_stored":56623104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53489053696,"internally_reserved":0,"allocated":63819776,"data_stored":63819776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53492002816,"internally_reserved":0,"allocated":60870656,"data_stored":60870656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53479510016,"internally_reserved":0,"allocated":73363456,"data_stored":73363456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:47.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 5820 5804 1.51133 1.42969 0.0306073 0.0824701 2024-09-22T03:23:48.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 6002 5986 1.50844 1.42188 0.104289 0.0826405 2024-09-22T03:23:48.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:48.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:48.289+0000 7fd432397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:48.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:48.295+0000 7fd432397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:49.048 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 16 6186 6170 1.50622 1.4375 0.0760838 0.0828042 2024-09-22T03:23:50.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 6386 6370 1.50792 1.5625 0.10216 0.0827526 2024-09-22T03:23:50.528 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:50.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:50.645+0000 7f840bd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:50.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:50.650+0000 7f840bd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:51.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 6558 6542 1.50309 1.34375 0.0578819 0.0830325 2024-09-22T03:23:51.458 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:51.458 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:51.458 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:23:51.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:51.577+0000 7f3e68ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:51.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:51.586+0000 7f3e68ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:51.818 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:23:51.819 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:23:51.839 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-22T03:23:49.944852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48087603,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5937,"num_write_kb":48280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5882,"ondisk_log_size":5882,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":292676,"kb_used_data":292676,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208898236,"statfs":{"total":214211493888,"available":213911793664,"internally_reserved":0,"allocated":299700224,"data_stored":299700224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15097856,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1843,"num_write_kb":14744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002074"},"pg_stats":[{"pgid":"3.f","version":"23'359","reported_seq":384,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.524554+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:23:46.524554+0000","last_peered":"2024-09-22T03:23:46.524554+0000","last_clean":"2024-09-22T03:23:46.524554+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:23:46.524554+0000","last_undegraded":"2024-09-22T03:23:46.524554+0000","last_fullsized":"2024-09-22T03:23:46.524554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'407","reported_seq":432,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:44.765678+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:23:44.765678+0000","last_peered":"2024-09-22T03:23:44.765678+0000","last_clean":"2024-09-22T03:23:44.765678+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:23:44.765678+0000","last_undegraded":"2024-09-22T03:23:44.765678+0000","last_fullsized":"2024-09-22T03:23:44.765678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'359","reported_seq":384,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.598981+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:23:46.598981+0000","last_peered":"2024-09-22T03:23:46.598981+0000","last_clean":"2024-09-22T03:23:46.598981+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:23:46.598981+0000","last_undegraded":"2024-09-22T03:23:46.598981+0000","last_fullsized":"2024-09-22T03:23:46.598981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'327","reported_seq":352,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.875251+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:23:46.875251+0000","last_peered":"2024-09-22T03:23:46.875251+0000","last_clean":"2024-09-22T03:23:46.875251+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:23:46.875251+0000","last_undegraded":"2024-09-22T03:23:46.875251+0000","last_fullsized":"2024-09-22T03:23:46.875251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'343","reported_seq":368,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.667517+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:23:46.667517+0000","last_peered":"2024-09-22T03:23:46.667517+0000","last_clean":"2024-09-22T03:23:46.667517+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:23:46.667517+0000","last_undegraded":"2024-09-22T03:23:46.667517+0000","last_fullsized":"2024-09-22T03:23:46.667517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'343","reported_seq":368,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:45.233051+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:23:45.233051+0000","last_peered":"2024-09-22T03:23:45.233051+0000","last_clean":"2024-09-22T03:23:45.233051+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:23:45.233051+0000","last_undegraded":"2024-09-22T03:23:45.233051+0000","last_fullsized":"2024-09-22T03:23:45.233051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"23'375","reported_seq":400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:45.999714+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:23:45.999714+0000","last_peered":"2024-09-22T03:23:45.999714+0000","last_clean":"2024-09-22T03:23:45.999714+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:23:45.999714+0000","last_undegraded":"2024-09-22T03:23:45.999714+0000","last_fullsized":"2024-09-22T03:23:45.999714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'343","reported_seq":368,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:45.605251+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:23:45.605251+0000","last_peered":"2024-09-22T03:23:45.605251+0000","last_clean":"2024-09-22T03:23:45.605251+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:23:45.605251+0000","last_undegraded":"2024-09-22T03:23:45.605251+0000","last_fullsized":"2024-09-22T03:23:45.605251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.026474+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:29.026474+0000","last_peered":"2024-09-22T03:23:29.026474+0000","last_clean":"2024-09-22T03:23:29.026474+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:29.026474+0000","last_undegraded":"2024-09-22T03:23:29.026474+0000","last_fullsized":"2024-09-22T03:23:29.026474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.055016+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:29.055016+0000","last_peered":"2024-09-22T03:23:29.055016+0000","last_clean":"2024-09-22T03:23:29.055016+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:29.055016+0000","last_undegraded":"2024-09-22T03:23:29.055016+0000","last_fullsized":"2024-09-22T03:23:29.055016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'311","reported_seq":336,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.424308+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:23:46.424308+0000","last_peered":"2024-09-22T03:23:46.424308+0000","last_clean":"2024-09-22T03:23:46.424308+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:23:46.424308+0000","last_undegraded":"2024-09-22T03:23:46.424308+0000","last_fullsized":"2024-09-22T03:23:46.424308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.047802+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:29.047802+0000","last_peered":"2024-09-22T03:23:29.047802+0000","last_clean":"2024-09-22T03:23:29.047802+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:29.047802+0000","last_undegraded":"2024-09-22T03:23:29.047802+0000","last_fullsized":"2024-09-22T03:23:29.047802+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'325","reported_seq":350,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.934402+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:23:46.934402+0000","last_peered":"2024-09-22T03:23:46.934402+0000","last_clean":"2024-09-22T03:23:46.934402+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:23:46.934402+0000","last_undegraded":"2024-09-22T03:23:46.934402+0000","last_fullsized":"2024-09-22T03:23:46.934402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2670592,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":326,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.051712+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.051712+0000","last_peered":"2024-09-22T03:23:29.051712+0000","last_clean":"2024-09-22T03:23:29.051712+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:29.051712+0000","last_undegraded":"2024-09-22T03:23:29.051712+0000","last_fullsized":"2024-09-22T03:23:29.051712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'391","reported_seq":416,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.650008+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:23:46.650008+0000","last_peered":"2024-09-22T03:23:46.650008+0000","last_clean":"2024-09-22T03:23:46.650008+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:23:46.650008+0000","last_undegraded":"2024-09-22T03:23:46.650008+0000","last_fullsized":"2024-09-22T03:23:46.650008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.043472+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.043472+0000","last_peered":"2024-09-22T03:23:29.043472+0000","last_clean":"2024-09-22T03:23:29.043472+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:29.043472+0000","last_undegraded":"2024-09-22T03:23:29.043472+0000","last_fullsized":"2024-09-22T03:23:29.043472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'295","reported_seq":320,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.032250+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:23:46.032250+0000","last_peered":"2024-09-22T03:23:46.032250+0000","last_clean":"2024-09-22T03:23:46.032250+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:23:46.032250+0000","last_undegraded":"2024-09-22T03:23:46.032250+0000","last_fullsized":"2024-09-22T03:23:46.032250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'367","reported_seq":392,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:44.139561+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:23:44.139561+0000","last_peered":"2024-09-22T03:23:44.139561+0000","last_clean":"2024-09-22T03:23:44.139561+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:23:44.139561+0000","last_undegraded":"2024-09-22T03:23:44.139561+0000","last_fullsized":"2024-09-22T03:23:44.139561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.039397+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.039397+0000","last_peered":"2024-09-22T03:23:29.039397+0000","last_clean":"2024-09-22T03:23:29.039397+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:29.039397+0000","last_undegraded":"2024-09-22T03:23:29.039397+0000","last_fullsized":"2024-09-22T03:23:29.039397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'463","reported_seq":488,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.559481+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:23:46.559481+0000","last_peered":"2024-09-22T03:23:46.559481+0000","last_clean":"2024-09-22T03:23:46.559481+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:23:46.559481+0000","last_undegraded":"2024-09-22T03:23:46.559481+0000","last_fullsized":"2024-09-22T03:23:46.559481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.028966+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.028966+0000","last_peered":"2024-09-22T03:23:29.028966+0000","last_clean":"2024-09-22T03:23:29.028966+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:29.028966+0000","last_undegraded":"2024-09-22T03:23:29.028966+0000","last_fullsized":"2024-09-22T03:23:29.028966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'319","reported_seq":344,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.487579+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:23:46.487579+0000","last_peered":"2024-09-22T03:23:46.487579+0000","last_clean":"2024-09-22T03:23:46.487579+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:23:46.487579+0000","last_undegraded":"2024-09-22T03:23:46.487579+0000","last_fullsized":"2024-09-22T03:23:46.487579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.036076+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:29.036076+0000","last_peered":"2024-09-22T03:23:29.036076+0000","last_clean":"2024-09-22T03:23:29.036076+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:29.036076+0000","last_undegraded":"2024-09-22T03:23:29.036076+0000","last_fullsized":"2024-09-22T03:23:29.036076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'455","reported_seq":480,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:46.848578+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:23:46.848578+0000","last_peered":"2024-09-22T03:23:46.848578+0000","last_clean":"2024-09-22T03:23:46.848578+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:23:46.848578+0000","last_undegraded":"2024-09-22T03:23:46.848578+0000","last_fullsized":"2024-09-22T03:23:46.848578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.058166+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:29.058166+0000","last_peered":"2024-09-22T03:23:29.058166+0000","last_clean":"2024-09-22T03:23:29.058166+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:29.058166+0000","last_undegraded":"2024-09-22T03:23:29.058166+0000","last_fullsized":"2024-09-22T03:23:29.058166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":47497216,"num_objects":725,"num_object_clones":0,"num_object_copies":2175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5798,"num_write_kb":46384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213910695936,"internally_reserved":0,"allocated":300797952,"data_stored":300797952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5782,"ondisk_log_size":5782,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213910695936,"internally_reserved":0,"allocated":300797952,"data_stored":300797952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53465468928,"internally_reserved":0,"allocated":87404544,"data_stored":87404544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":85116,"kb_used_data":85116,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52212612,"statfs":{"total":53552873472,"available":53465714688,"internally_reserved":0,"allocated":87158784,"data_stored":87158784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":69700,"kb_used_data":69700,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52228028,"statfs":{"total":53552873472,"available":53481500672,"internally_reserved":0,"allocated":71372800,"data_stored":71372800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":72964,"kb_used_data":72964,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52224764,"statfs":{"total":53552873472,"available":53478158336,"internally_reserved":0,"allocated":74715136,"data_stored":74715136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":64896,"kb_used_data":64896,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52232832,"statfs":{"total":53552873472,"available":53486419968,"internally_reserved":0,"allocated":66453504,"data_stored":66453504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53465468928,"internally_reserved":0,"allocated":87404544,"data_stored":87404544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53486182400,"internally_reserved":0,"allocated":66691072,"data_stored":66691072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53477863424,"internally_reserved":0,"allocated":75010048,"data_stored":75010048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53481181184,"internally_reserved":0,"allocated":71692288,"data_stored":71692288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53465468928,"internally_reserved":0,"allocated":87404544,"data_stored":87404544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53486182400,"internally_reserved":0,"allocated":66691072,"data_stored":66691072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53477863424,"internally_reserved":0,"allocated":75010048,"data_stored":75010048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53481181184,"internally_reserved":0,"allocated":71692288,"data_stored":71692288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53465468928,"internally_reserved":0,"allocated":87404544,"data_stored":87404544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:23:52.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 6754 6738 1.50389 1.53125 0.0804421 0.0830123 2024-09-22T03:23:52.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:52.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:52.919+0000 7f92c7b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:52.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:52.927+0000 7f92c7b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:53.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 6937 6921 1.50182 1.42969 0.091321 0.0830569 2024-09-22T03:23:54.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 7110 7094 1.49776 1.35156 0.0474395 0.0833369 2024-09-22T03:23:55.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 7302 7286 1.49781 1.5 0.148938 0.0832471 2024-09-22T03:23:55.125 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:55.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:55.259+0000 7efde7d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:55.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:55.265+0000 7efde7d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:56.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 7502 7486 1.49947 1.5625 0.0269116 0.0832363 2024-09-22T03:23:56.841 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:23:56.841 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:23:56.841 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:23:56.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:23:56.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:56.971+0000 7fb697a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:56.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:56.983+0000 7fb697a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:56.998 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:56.999 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:57.002 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:57.002 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:57.002 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:23:57.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:23:57.049050+0000 min lat: 0.0101625 max lat: 0.317369 avg lat: 0.0834804 2024-09-22T03:23:57.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:23:57.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 7674 7658 1.49557 1.34375 0.0397753 0.0834804 2024-09-22T03:23:57.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.139+0000 7fc8cb157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:57.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.150+0000 7fc8cb157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:57.173 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:57.173 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:57.174 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:57.176 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:57.177 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:57.177 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:23:57.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.293+0000 7f45896d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:57.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.302+0000 7f45896d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:57.317 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:57.320 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:57.320 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:57.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:23:57.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:23:57.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.466+0000 7fe652b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:57.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.479+0000 7fe652b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:23:57.532 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:23:57.536 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:23:57.536 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:23:57.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.578+0000 7f1c85157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:57.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:57.584+0000 7f1c85157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:58.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 7858 7842 1.49415 1.4375 0.0621495 0.0835676 2024-09-22T03:23:59.049 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 8040 8024 1.49243 1.42188 0.105505 0.0836473 2024-09-22T03:23:59.735 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:23:59.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:59.840+0000 7f69fbd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:23:59.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:23:59.845+0000 7f69fbd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:00.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 16 8190 8174 1.48497 1.17188 0.0500495 0.0840191 2024-09-22T03:24:01.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 8375 8359 1.48407 1.44531 0.0322336 0.0841453 2024-09-22T03:24:01.967 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:02.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 8554 8538 1.48216 1.39844 0.0452641 0.0842222 2024-09-22T03:24:02.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:02.099+0000 7f012d30e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:02.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:02.106+0000 7f012d30e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:02.537 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:02.538 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:02.538 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:24:02.538 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:24:02.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:02.674+0000 7f77c706c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:02.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:02.684+0000 7f77c706c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:02.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:02.714 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:02.716 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:02.716 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:02.716 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:24:02.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:02.869+0000 7f5e77a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:02.880+0000 7f5e77a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:02.896 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:02.897 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:02.897 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:02.897 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:02.897 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:02.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:02.901 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:02.901 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:24:03.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:03.029+0000 7f1289397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:03.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:03.036+0000 7f1289397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:03.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 8712 8696 1.47677 1.23438 0.134635 0.0844448 2024-09-22T03:24:03.051 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:03.051 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:03.051 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:03.051 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:03.051 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:03.051 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:03.052 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:03.052 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:03.052 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:03.052 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:03.052 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:03.052 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:03.054 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:03.054 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:03.055 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:24:03.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:03.175+0000 7fa468d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:03.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:03.184+0000 7fa468d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:03.204 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:03.207 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:03.208 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:04.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 16 8869 8853 1.47145 1.22656 0.0905968 0.0848394 2024-09-22T03:24:04.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:04.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:04.404+0000 7feff306c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:04.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:04.410+0000 7feff306c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:05.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 9035 9019 1.46781 1.29688 0.0570108 0.0851027 2024-09-22T03:24:06.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 9229 9213 1.46878 1.51562 0.155897 0.0850159 2024-09-22T03:24:06.541 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:06.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:06.651+0000 7f0d4df17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:06.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:06.655+0000 7f0d4df17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:07.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 9389 9373 1.4644 1.25 0.130013 0.0852819 2024-09-22T03:24:08.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 9587 9571 1.46602 1.54688 0.0344309 0.0851826 2024-09-22T03:24:08.209 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:08.209 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:08.209 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:08.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:08.323+0000 7f4265c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:08.330+0000 7f4265c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:24:08.638 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:24:08.660 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-22T03:24:07.948165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77308467,"num_objects":1175,"num_object_clones":0,"num_object_copies":3525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9504,"num_write_kb":76816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9449,"ondisk_log_size":9449,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":470948,"kb_used_data":470948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208719964,"statfs":{"total":214211493888,"available":213729243136,"internally_reserved":0,"allocated":482250752,"data_stored":482250752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21577728,"num_objects":329,"num_object_clones":0,"num_object_copies":987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2634,"num_write_kb":21072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002184"},"pg_stats":[{"pgid":"3.f","version":"23'623","reported_seq":648,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.133032+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:24:06.133032+0000","last_peered":"2024-09-22T03:24:06.133032+0000","last_clean":"2024-09-22T03:24:06.133032+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:24:06.133032+0000","last_undegraded":"2024-09-22T03:24:06.133032+0000","last_fullsized":"2024-09-22T03:24:06.133032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'607","reported_seq":632,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.863169+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:24:06.863169+0000","last_peered":"2024-09-22T03:24:06.863169+0000","last_clean":"2024-09-22T03:24:06.863169+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:24:06.863169+0000","last_undegraded":"2024-09-22T03:24:06.863169+0000","last_fullsized":"2024-09-22T03:24:06.863169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'519","reported_seq":544,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:04.821848+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:24:04.821848+0000","last_peered":"2024-09-22T03:24:04.821848+0000","last_clean":"2024-09-22T03:24:04.821848+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:24:04.821848+0000","last_undegraded":"2024-09-22T03:24:04.821848+0000","last_fullsized":"2024-09-22T03:24:04.821848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'455","reported_seq":480,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.514692+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:24:06.514692+0000","last_peered":"2024-09-22T03:24:06.514692+0000","last_clean":"2024-09-22T03:24:06.514692+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:24:06.514692+0000","last_undegraded":"2024-09-22T03:24:06.514692+0000","last_fullsized":"2024-09-22T03:24:06.514692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'599","reported_seq":624,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.918568+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:24:06.918568+0000","last_peered":"2024-09-22T03:24:06.918568+0000","last_clean":"2024-09-22T03:24:06.918568+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:24:06.918568+0000","last_undegraded":"2024-09-22T03:24:06.918568+0000","last_fullsized":"2024-09-22T03:24:06.918568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'631","reported_seq":656,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:05.132762+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:24:05.132762+0000","last_peered":"2024-09-22T03:24:05.132762+0000","last_clean":"2024-09-22T03:24:05.132762+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:24:05.132762+0000","last_undegraded":"2024-09-22T03:24:05.132762+0000","last_fullsized":"2024-09-22T03:24:05.132762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"23'580","reported_seq":605,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.941163+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:24:06.941163+0000","last_peered":"2024-09-22T03:24:06.941163+0000","last_clean":"2024-09-22T03:24:06.941163+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:24:06.941163+0000","last_undegraded":"2024-09-22T03:24:06.941163+0000","last_fullsized":"2024-09-22T03:24:06.941163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4759552,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":581,"num_write_kb":4648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'567","reported_seq":592,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.744814+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:24:06.744814+0000","last_peered":"2024-09-22T03:24:06.744814+0000","last_clean":"2024-09-22T03:24:06.744814+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:24:06.744814+0000","last_undegraded":"2024-09-22T03:24:06.744814+0000","last_fullsized":"2024-09-22T03:24:06.744814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.026474+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:29.026474+0000","last_peered":"2024-09-22T03:23:29.026474+0000","last_clean":"2024-09-22T03:23:29.026474+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:29.026474+0000","last_undegraded":"2024-09-22T03:23:29.026474+0000","last_fullsized":"2024-09-22T03:23:29.026474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.055016+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:29.055016+0000","last_peered":"2024-09-22T03:23:29.055016+0000","last_clean":"2024-09-22T03:23:29.055016+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:29.055016+0000","last_undegraded":"2024-09-22T03:23:29.055016+0000","last_fullsized":"2024-09-22T03:23:29.055016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'543","reported_seq":568,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.099304+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:24:06.099304+0000","last_peered":"2024-09-22T03:24:06.099304+0000","last_clean":"2024-09-22T03:24:06.099304+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:24:06.099304+0000","last_undegraded":"2024-09-22T03:24:06.099304+0000","last_fullsized":"2024-09-22T03:24:06.099304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.047802+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:29.047802+0000","last_peered":"2024-09-22T03:23:29.047802+0000","last_clean":"2024-09-22T03:23:29.047802+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:29.047802+0000","last_undegraded":"2024-09-22T03:23:29.047802+0000","last_fullsized":"2024-09-22T03:23:29.047802+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'575","reported_seq":600,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.387771+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:24:06.387771+0000","last_peered":"2024-09-22T03:24:06.387771+0000","last_clean":"2024-09-22T03:24:06.387771+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:24:06.387771+0000","last_undegraded":"2024-09-22T03:24:06.387771+0000","last_fullsized":"2024-09-22T03:24:06.387771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.051712+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.051712+0000","last_peered":"2024-09-22T03:23:29.051712+0000","last_clean":"2024-09-22T03:23:29.051712+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:29.051712+0000","last_undegraded":"2024-09-22T03:23:29.051712+0000","last_fullsized":"2024-09-22T03:23:29.051712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'607","reported_seq":632,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:05.399631+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:24:05.399631+0000","last_peered":"2024-09-22T03:24:05.399631+0000","last_clean":"2024-09-22T03:24:05.399631+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:24:05.399631+0000","last_undegraded":"2024-09-22T03:24:05.399631+0000","last_fullsized":"2024-09-22T03:24:05.399631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.043472+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.043472+0000","last_peered":"2024-09-22T03:23:29.043472+0000","last_clean":"2024-09-22T03:23:29.043472+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:29.043472+0000","last_undegraded":"2024-09-22T03:23:29.043472+0000","last_fullsized":"2024-09-22T03:23:29.043472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'479","reported_seq":504,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.342355+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:24:06.342355+0000","last_peered":"2024-09-22T03:24:06.342355+0000","last_clean":"2024-09-22T03:24:06.342355+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:24:06.342355+0000","last_undegraded":"2024-09-22T03:24:06.342355+0000","last_fullsized":"2024-09-22T03:24:06.342355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'567","reported_seq":592,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.555041+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:24:06.555041+0000","last_peered":"2024-09-22T03:24:06.555041+0000","last_clean":"2024-09-22T03:24:06.555041+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:24:06.555041+0000","last_undegraded":"2024-09-22T03:24:06.555041+0000","last_fullsized":"2024-09-22T03:24:06.555041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.039397+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.039397+0000","last_peered":"2024-09-22T03:23:29.039397+0000","last_clean":"2024-09-22T03:23:29.039397+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:29.039397+0000","last_undegraded":"2024-09-22T03:23:29.039397+0000","last_fullsized":"2024-09-22T03:23:29.039397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'751","reported_seq":776,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:05.618770+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:24:05.618770+0000","last_peered":"2024-09-22T03:24:05.618770+0000","last_clean":"2024-09-22T03:24:05.618770+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:24:05.618770+0000","last_undegraded":"2024-09-22T03:24:05.618770+0000","last_fullsized":"2024-09-22T03:24:05.618770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.028966+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.028966+0000","last_peered":"2024-09-22T03:23:29.028966+0000","last_clean":"2024-09-22T03:23:29.028966+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:29.028966+0000","last_undegraded":"2024-09-22T03:23:29.028966+0000","last_fullsized":"2024-09-22T03:23:29.028966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'511","reported_seq":536,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:05.696444+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:24:05.696444+0000","last_peered":"2024-09-22T03:24:05.696444+0000","last_clean":"2024-09-22T03:24:05.696444+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:24:05.696444+0000","last_undegraded":"2024-09-22T03:24:05.696444+0000","last_fullsized":"2024-09-22T03:24:05.696444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.036076+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:29.036076+0000","last_peered":"2024-09-22T03:23:29.036076+0000","last_clean":"2024-09-22T03:23:29.036076+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:29.036076+0000","last_undegraded":"2024-09-22T03:23:29.036076+0000","last_fullsized":"2024-09-22T03:23:29.036076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'735","reported_seq":760,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:06.802256+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:24:06.802256+0000","last_peered":"2024-09-22T03:24:06.802256+0000","last_clean":"2024-09-22T03:24:06.802256+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:24:06.802256+0000","last_undegraded":"2024-09-22T03:24:06.802256+0000","last_fullsized":"2024-09-22T03:24:06.802256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.058166+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:29.058166+0000","last_peered":"2024-09-22T03:23:29.058166+0000","last_clean":"2024-09-22T03:23:29.058166+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:29.058166+0000","last_undegraded":"2024-09-22T03:23:29.058166+0000","last_fullsized":"2024-09-22T03:23:29.058166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":76718080,"num_objects":1171,"num_object_clones":0,"num_object_copies":3513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9365,"num_write_kb":74920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213728002048,"internally_reserved":0,"allocated":483491840,"data_stored":483491840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9349,"ondisk_log_size":9349,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213728010240,"internally_reserved":0,"allocated":483483648,"data_stored":483483648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53412732928,"internally_reserved":0,"allocated":140140544,"data_stored":140140544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":136520,"kb_used_data":136520,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52161208,"statfs":{"total":53552873472,"available":53413076992,"internally_reserved":0,"allocated":139796480,"data_stored":139796480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":110820,"kb_used_data":110820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52186908,"statfs":{"total":53552873472,"available":53439393792,"internally_reserved":0,"allocated":113479680,"data_stored":113479680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":117964,"kb_used_data":117964,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52179764,"statfs":{"total":53552873472,"available":53432078336,"internally_reserved":0,"allocated":120795136,"data_stored":120795136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":105644,"kb_used_data":105644,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52192084,"statfs":{"total":53552873472,"available":53444694016,"internally_reserved":0,"allocated":108179456,"data_stored":108179456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53412732928,"internally_reserved":0,"allocated":140140544,"data_stored":140140544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53444517888,"internally_reserved":0,"allocated":108355584,"data_stored":108355584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53431701504,"internally_reserved":0,"allocated":121171968,"data_stored":121171968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53439057920,"internally_reserved":0,"allocated":113815552,"data_stored":113815552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53412732928,"internally_reserved":0,"allocated":140140544,"data_stored":140140544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53444509696,"internally_reserved":0,"allocated":108363776,"data_stored":108363776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53431701504,"internally_reserved":0,"allocated":121171968,"data_stored":121171968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53439057920,"internally_reserved":0,"allocated":113815552,"data_stored":113815552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53412732928,"internally_reserved":0,"allocated":140140544,"data_stored":140140544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:08.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:08.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:08.872+0000 7fc85a86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:08.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:08.880+0000 7fc85a86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:09.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 9748 9732 1.46201 1.25781 0.0975489 0.085331 2024-09-22T03:24:10.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 9935 9919 1.46199 1.46094 0.0516619 0.0854213 2024-09-22T03:24:11.024 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:11.050 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 10090 10074 1.45734 1.21094 0.0677467 0.0857064 2024-09-22T03:24:11.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:11.133+0000 7fc756ea9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:11.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:11.138+0000 7fc756ea9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:12.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 10268 10252 1.45612 1.39062 0.0850294 0.0857762 2024-09-22T03:24:13.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 10450 10434 1.45551 1.42188 0.0600318 0.0858241 2024-09-22T03:24:13.261 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:13.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:13.375+0000 7fe0a9797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:13.381+0000 7fe0a9797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.662 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:13.662 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:13.662 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:24:13.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:24:13.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:13.779+0000 7f9265997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:13.787+0000 7f9265997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.823 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:13.824 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:13.827 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:13.828 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:13.828 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:24:13.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:13.979+0000 7f5ddfcd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:13.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:13.989+0000 7f5ddfcd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:14.020 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:14.020 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:14.020 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:14.020 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:14.021 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:14.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:14.024 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:14.024 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:24:14.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 10615 10599 1.45259 1.28906 0.0687301 0.08593 2024-09-22T03:24:14.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:14.137+0000 7f276362c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:14.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:14.146+0000 7f276362c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:14.191 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:14.191 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:14.192 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:14.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:14.195 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:14.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:24:14.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:14.350+0000 7fcec3b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:14.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:14.359+0000 7fcec3b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:14.376 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:14.377 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:14.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:14.380 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:15.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 10793 10777 1.45152 1.39062 0.0521045 0.0860618 2024-09-22T03:24:15.493 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:15.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:15.630+0000 7f3175997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:15.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:15.636+0000 7f3175997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:16.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 10979 10963 1.45154 1.45312 0.0744321 0.0860221 2024-09-22T03:24:17.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:24:17.050848+0000 min lat: 0.0101625 max lat: 0.317369 avg lat: 0.0861251 2024-09-22T03:24:17.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:24:17.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 11155 11139 1.45026 1.375 0.0906147 0.0861251 2024-09-22T03:24:17.801 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:17.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:17.919+0000 7fcf91757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:17.925+0000 7fcf91757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:18.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 11339 11323 1.45005 1.4375 0.0280422 0.0861211 2024-09-22T03:24:19.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 11516 11500 1.44896 1.38281 0.073234 0.0861448 2024-09-22T03:24:19.381 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:19.381 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:19.382 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:24:19.382 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:24:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:19.494+0000 7f38ba197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:19.506+0000 7f38ba197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:19.528 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:19.529 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:19.529 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:24:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:19.652+0000 7f3d21797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:19.663+0000 7f3d21797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.708 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:19.708 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:19.708 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:19.708 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:19.708 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:19.708 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:19.708 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:19.709 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:19.709 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:19.709 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:19.709 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:19.709 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:19.711 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:19.711 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:19.712 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:24:19.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:19.852+0000 7f5ca9797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:19.862+0000 7f5ca9797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:19.906 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:19.906 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:19.907 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:19.907 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:19.907 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:19.907 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:19.907 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:19.907 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:19.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:19.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:19.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:19.908 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:19.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:19.910 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:19.910 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:24:20.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:20.019+0000 7f6bb9f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:20.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:20.029+0000 7f6bb9f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:20.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 11688 11672 1.44729 1.34375 0.117303 0.0862444 2024-09-22T03:24:20.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:20.102 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:20.102 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:20.102 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:20.106 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:20.107 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:20.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:20.222+0000 7f3b02f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:20.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:20.228+0000 7f3b02f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:21.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 16 11872 11856 1.44714 1.4375 0.0917099 0.0863093 2024-09-22T03:24:22.051 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 12059 12043 1.44735 1.46094 0.0611279 0.0863113 2024-09-22T03:24:22.403 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:22.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:22.541+0000 7f9fc522c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:22.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:22.547+0000 7f9fc522c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:23.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 12218 12202 1.44424 1.24219 0.0773506 0.0864887 2024-09-22T03:24:24.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 12366 12350 1.43994 1.15625 0.0303198 0.0866547 2024-09-22T03:24:24.695 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:24.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:24.833+0000 7f50d4c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:24.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:24.839+0000 7f50d4c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:25.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 12547 12531 1.43956 1.41406 0.0595198 0.0867862 2024-09-22T03:24:25.107 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:25.107 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:25.108 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:25.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:25.217+0000 7fdf0eb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:25.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:25.224+0000 7fdf0eb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:25.432 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:24:25.433 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:24:25.449 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-22T03:24:23.950847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":99140147,"num_objects":1508,"num_object_clones":0,"num_object_copies":4524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12169,"num_write_kb":98136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12114,"ondisk_log_size":12114,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":604432,"kb_used_data":604432,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208586480,"statfs":{"total":214211493888,"available":213592555520,"internally_reserved":0,"allocated":618938368,"data_stored":618938368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21831680,"num_objects":333,"num_object_clones":0,"num_object_copies":999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2665,"num_write_kb":21320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001880"},"pg_stats":[{"pgid":"3.f","version":"23'783","reported_seq":808,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.647161+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:24:21.647161+0000","last_peered":"2024-09-22T03:24:21.647161+0000","last_clean":"2024-09-22T03:24:21.647161+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:24:21.647161+0000","last_undegraded":"2024-09-22T03:24:21.647161+0000","last_fullsized":"2024-09-22T03:24:21.647161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'743","reported_seq":768,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.776261+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:24:21.776261+0000","last_peered":"2024-09-22T03:24:21.776261+0000","last_clean":"2024-09-22T03:24:21.776261+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:24:21.776261+0000","last_undegraded":"2024-09-22T03:24:21.776261+0000","last_fullsized":"2024-09-22T03:24:21.776261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'695","reported_seq":720,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.329131+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:24:21.329131+0000","last_peered":"2024-09-22T03:24:21.329131+0000","last_clean":"2024-09-22T03:24:21.329131+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:24:21.329131+0000","last_undegraded":"2024-09-22T03:24:21.329131+0000","last_fullsized":"2024-09-22T03:24:21.329131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'647","reported_seq":672,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:20.967511+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:24:20.967511+0000","last_peered":"2024-09-22T03:24:20.967511+0000","last_clean":"2024-09-22T03:24:20.967511+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:24:20.967511+0000","last_undegraded":"2024-09-22T03:24:20.967511+0000","last_fullsized":"2024-09-22T03:24:20.967511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'735","reported_seq":760,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.524679+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:24:21.524679+0000","last_peered":"2024-09-22T03:24:21.524679+0000","last_clean":"2024-09-22T03:24:21.524679+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:24:21.524679+0000","last_undegraded":"2024-09-22T03:24:21.524679+0000","last_fullsized":"2024-09-22T03:24:21.524679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'775","reported_seq":800,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:20.216945+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:24:20.216945+0000","last_peered":"2024-09-22T03:24:20.216945+0000","last_clean":"2024-09-22T03:24:20.216945+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:24:20.216945+0000","last_undegraded":"2024-09-22T03:24:20.216945+0000","last_fullsized":"2024-09-22T03:24:20.216945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"23'767","reported_seq":792,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.451614+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:24:21.451614+0000","last_peered":"2024-09-22T03:24:21.451614+0000","last_clean":"2024-09-22T03:24:21.451614+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:24:21.451614+0000","last_undegraded":"2024-09-22T03:24:21.451614+0000","last_fullsized":"2024-09-22T03:24:21.451614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'751","reported_seq":776,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:19.981284+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:24:19.981284+0000","last_peered":"2024-09-22T03:24:19.981284+0000","last_clean":"2024-09-22T03:24:19.981284+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:24:19.981284+0000","last_undegraded":"2024-09-22T03:24:19.981284+0000","last_fullsized":"2024-09-22T03:24:19.981284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.026474+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:23:29.026474+0000","last_peered":"2024-09-22T03:23:29.026474+0000","last_clean":"2024-09-22T03:23:29.026474+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:23:29.026474+0000","last_undegraded":"2024-09-22T03:23:29.026474+0000","last_fullsized":"2024-09-22T03:23:29.026474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.055016+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:23:29.055016+0000","last_peered":"2024-09-22T03:23:29.055016+0000","last_clean":"2024-09-22T03:23:29.055016+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:23:29.055016+0000","last_undegraded":"2024-09-22T03:23:29.055016+0000","last_fullsized":"2024-09-22T03:23:29.055016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'759","reported_seq":784,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.493677+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:24:21.493677+0000","last_peered":"2024-09-22T03:24:21.493677+0000","last_clean":"2024-09-22T03:24:21.493677+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:24:21.493677+0000","last_undegraded":"2024-09-22T03:24:21.493677+0000","last_fullsized":"2024-09-22T03:24:21.493677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.047802+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:23:29.047802+0000","last_peered":"2024-09-22T03:23:29.047802+0000","last_clean":"2024-09-22T03:23:29.047802+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:23:29.047802+0000","last_undegraded":"2024-09-22T03:23:29.047802+0000","last_fullsized":"2024-09-22T03:23:29.047802+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'679","reported_seq":704,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.222014+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:24:21.222014+0000","last_peered":"2024-09-22T03:24:21.222014+0000","last_clean":"2024-09-22T03:24:21.222014+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:24:21.222014+0000","last_undegraded":"2024-09-22T03:24:21.222014+0000","last_fullsized":"2024-09-22T03:24:21.222014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.051712+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.051712+0000","last_peered":"2024-09-22T03:23:29.051712+0000","last_clean":"2024-09-22T03:23:29.051712+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:23:29.051712+0000","last_undegraded":"2024-09-22T03:23:29.051712+0000","last_fullsized":"2024-09-22T03:23:29.051712+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'783","reported_seq":808,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.325251+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:24:21.325251+0000","last_peered":"2024-09-22T03:24:21.325251+0000","last_clean":"2024-09-22T03:24:21.325251+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:24:21.325251+0000","last_undegraded":"2024-09-22T03:24:21.325251+0000","last_fullsized":"2024-09-22T03:24:21.325251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.043472+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:23:29.043472+0000","last_peered":"2024-09-22T03:23:29.043472+0000","last_clean":"2024-09-22T03:23:29.043472+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:23:29.043472+0000","last_undegraded":"2024-09-22T03:23:29.043472+0000","last_fullsized":"2024-09-22T03:23:29.043472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'669","reported_seq":694,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.942062+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:24:21.942062+0000","last_peered":"2024-09-22T03:24:21.942062+0000","last_clean":"2024-09-22T03:24:21.942062+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:24:21.942062+0000","last_undegraded":"2024-09-22T03:24:21.942062+0000","last_fullsized":"2024-09-22T03:24:21.942062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5488640,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":670,"num_write_kb":5360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'727","reported_seq":752,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.267962+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:24:21.267962+0000","last_peered":"2024-09-22T03:24:21.267962+0000","last_clean":"2024-09-22T03:24:21.267962+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:24:21.267962+0000","last_undegraded":"2024-09-22T03:24:21.267962+0000","last_fullsized":"2024-09-22T03:24:21.267962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.039397+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.039397+0000","last_peered":"2024-09-22T03:23:29.039397+0000","last_clean":"2024-09-22T03:23:29.039397+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:23:29.039397+0000","last_undegraded":"2024-09-22T03:23:29.039397+0000","last_fullsized":"2024-09-22T03:23:29.039397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'911","reported_seq":936,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.065909+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:24:21.065909+0000","last_peered":"2024-09-22T03:24:21.065909+0000","last_clean":"2024-09-22T03:24:21.065909+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:24:21.065909+0000","last_undegraded":"2024-09-22T03:24:21.065909+0000","last_fullsized":"2024-09-22T03:24:21.065909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.028966+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:23:29.028966+0000","last_peered":"2024-09-22T03:23:29.028966+0000","last_clean":"2024-09-22T03:23:29.028966+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:23:29.028966+0000","last_undegraded":"2024-09-22T03:23:29.028966+0000","last_fullsized":"2024-09-22T03:23:29.028966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'711","reported_seq":736,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.906039+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:24:21.906039+0000","last_peered":"2024-09-22T03:24:21.906039+0000","last_clean":"2024-09-22T03:24:21.906039+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:24:21.906039+0000","last_undegraded":"2024-09-22T03:24:21.906039+0000","last_fullsized":"2024-09-22T03:24:21.906039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.036076+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:23:29.036076+0000","last_peered":"2024-09-22T03:23:29.036076+0000","last_clean":"2024-09-22T03:23:29.036076+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:23:29.036076+0000","last_undegraded":"2024-09-22T03:23:29.036076+0000","last_fullsized":"2024-09-22T03:23:29.036076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'879","reported_seq":904,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:24:21.725985+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:24:21.725985+0000","last_peered":"2024-09-22T03:24:21.725985+0000","last_clean":"2024-09-22T03:24:21.725985+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:24:21.725985+0000","last_undegraded":"2024-09-22T03:24:21.725985+0000","last_fullsized":"2024-09-22T03:24:21.725985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T03:23:29.058166+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:23:29.058166+0000","last_peered":"2024-09-22T03:23:29.058166+0000","last_clean":"2024-09-22T03:23:29.058166+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:23:29.058166+0000","last_undegraded":"2024-09-22T03:23:29.058166+0000","last_fullsized":"2024-09-22T03:23:29.058166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":98549760,"num_objects":1504,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12030,"num_write_kb":96240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213591703552,"internally_reserved":0,"allocated":619790336,"data_stored":619790336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12014,"ondisk_log_size":12014,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213591703552,"internally_reserved":0,"allocated":619790336,"data_stored":619790336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53373136896,"internally_reserved":0,"allocated":179736576,"data_stored":179736576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":175228,"kb_used_data":175228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52122500,"statfs":{"total":53552873472,"available":53373440000,"internally_reserved":0,"allocated":179433472,"data_stored":179433472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":141236,"kb_used_data":141236,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52156492,"statfs":{"total":53552873472,"available":53408247808,"internally_reserved":0,"allocated":144625664,"data_stored":144625664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":151236,"kb_used_data":151236,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52146492,"statfs":{"total":53552873472,"available":53398007808,"internally_reserved":0,"allocated":154865664,"data_stored":154865664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":136732,"kb_used_data":136732,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52160996,"statfs":{"total":53552873472,"available":53412859904,"internally_reserved":0,"allocated":140013568,"data_stored":140013568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53373136896,"internally_reserved":0,"allocated":179736576,"data_stored":179736576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53412614144,"internally_reserved":0,"allocated":140259328,"data_stored":140259328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53397819392,"internally_reserved":0,"allocated":155054080,"data_stored":155054080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53408133120,"internally_reserved":0,"allocated":144740352,"data_stored":144740352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53373136896,"internally_reserved":0,"allocated":179736576,"data_stored":179736576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53412614144,"internally_reserved":0,"allocated":140259328,"data_stored":140259328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53397819392,"internally_reserved":0,"allocated":155054080,"data_stored":155054080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53408133120,"internally_reserved":0,"allocated":144740352,"data_stored":144740352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53373136896,"internally_reserved":0,"allocated":179736576,"data_stored":179736576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:26.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 12705 12689 1.43658 1.23438 0.0742457 0.0869494 2024-09-22T03:24:26.983 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:27.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 12879 12863 1.43548 1.35938 0.0648071 0.086955 2024-09-22T03:24:27.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:27.089+0000 7f5b9dece640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:27.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:27.095+0000 7f5b9cc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:28.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 13056 13040 1.43473 1.38281 0.11869 0.0870303 2024-09-22T03:24:29.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 13216 13200 1.43217 1.25 0.174442 0.0871273 2024-09-22T03:24:29.252 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:29.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:29.375+0000 7fec03917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:29.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:29.381+0000 7fec03917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:30.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 13388 13372 1.43095 1.34375 0.107749 0.0872717 2024-09-22T03:24:30.451 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:30.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:24:30.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:30.572+0000 7fcd1ad57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:30.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:30.582+0000 7fcd1ad57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:31.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 13552 13536 1.42893 1.28125 0.0896498 0.0874191 2024-09-22T03:24:31.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:31.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:31.658+0000 7f5ad8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:31.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:31.665+0000 7f5ad8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:31.693 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:24:31.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:24:31.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:31.847+0000 7fd77002c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:31.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:31.863+0000 7fd77002c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:32.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 13710 13694 1.42633 1.23438 0.181946 0.0874845 2024-09-22T03:24:32.697 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:24:32.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:24:32.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:32.875+0000 7f815ccd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:32.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:32.885+0000 7f815ccd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:33.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 13870 13854 1.42401 1.25 0.0948747 0.0877279 2024-09-22T03:24:33.702 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:24:33.725 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:24:33.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:33.838+0000 7f1220757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:33.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:33.849+0000 7f1220757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:33.853 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:33.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:33.983+0000 7f1539797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:33.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:33.990+0000 7f1539797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:34.052 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 14014 13998 1.42013 1.125 0.072738 0.087931 2024-09-22T03:24:34.706 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:24:34.724 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:24:34.724 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:34.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:34.856+0000 7fe1bfca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:34.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:34.865+0000 7fe1bfca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:35.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 16 14177 14161 1.41824 1.27344 0.158587 0.0880674 2024-09-22T03:24:35.212 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:24:35.212 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:24:35.231 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-22T03:24:34.708192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112796211,"num_objects":1717,"num_object_clones":0,"num_object_copies":5151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13836,"num_write_kb":111472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13793,"ondisk_log_size":13793,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":687904,"kb_used_data":687904,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208503008,"statfs":{"total":214211493888,"available":213507080192,"internally_reserved":0,"allocated":704413696,"data_stored":704413696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6791168,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":6632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.756300"},"pg_stats":[{"pgid":"3.f","version":"23'896","reported_seq":923,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.721980+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:24:31.721980+0000","last_peered":"2024-09-22T03:24:31.721980+0000","last_clean":"2024-09-22T03:24:31.721980+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:24:31.721980+0000","last_undegraded":"2024-09-22T03:24:31.721980+0000","last_fullsized":"2024-09-22T03:24:31.721980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'832","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.714342+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:24:31.714342+0000","last_peered":"2024-09-22T03:24:31.714342+0000","last_clean":"2024-09-22T03:24:31.714342+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:24:31.714342+0000","last_undegraded":"2024-09-22T03:24:31.714342+0000","last_fullsized":"2024-09-22T03:24:31.714342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'784","reported_seq":811,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.790943+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:24:31.790943+0000","last_peered":"2024-09-22T03:24:31.790943+0000","last_clean":"2024-09-22T03:24:31.790943+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:24:31.790943+0000","last_undegraded":"2024-09-22T03:24:31.790943+0000","last_fullsized":"2024-09-22T03:24:31.790943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'800","reported_seq":827,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.719021+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:24:31.719021+0000","last_peered":"2024-09-22T03:24:31.719021+0000","last_clean":"2024-09-22T03:24:31.719021+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:24:31.719021+0000","last_undegraded":"2024-09-22T03:24:31.719021+0000","last_fullsized":"2024-09-22T03:24:31.719021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'831","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.910931+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:24:31.910931+0000","last_peered":"2024-09-22T03:24:31.910931+0000","last_clean":"2024-09-22T03:24:31.910931+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:24:31.910931+0000","last_undegraded":"2024-09-22T03:24:31.910931+0000","last_fullsized":"2024-09-22T03:24:31.910931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'872","reported_seq":899,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.716540+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:24:31.716540+0000","last_peered":"2024-09-22T03:24:31.716540+0000","last_clean":"2024-09-22T03:24:31.716540+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:24:31.716540+0000","last_undegraded":"2024-09-22T03:24:31.716540+0000","last_fullsized":"2024-09-22T03:24:31.716540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"24'880","reported_seq":908,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.858407+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:24:31.858407+0000","last_peered":"2024-09-22T03:24:31.858407+0000","last_clean":"2024-09-22T03:24:31.858407+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:24:31.858407+0000","last_undegraded":"2024-09-22T03:24:31.858407+0000","last_fullsized":"2024-09-22T03:24:31.858407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7217152,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":881,"num_write_kb":7048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'839","reported_seq":867,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.789216+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:24:31.789216+0000","last_peered":"2024-09-22T03:24:31.789216+0000","last_clean":"2024-09-22T03:24:31.789216+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:24:31.789216+0000","last_undegraded":"2024-09-22T03:24:31.789216+0000","last_fullsized":"2024-09-22T03:24:31.789216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881280,"num_objects":105,"num_object_clones":0,"num_object_copies":315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.755710+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:24:31.755710+0000","last_peered":"2024-09-22T03:24:31.755710+0000","last_clean":"2024-09-22T03:24:31.755710+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:24:31.755710+0000","last_undegraded":"2024-09-22T03:24:31.755710+0000","last_fullsized":"2024-09-22T03:24:31.755710+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":41,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.790279+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:24:31.790279+0000","last_peered":"2024-09-22T03:24:31.790279+0000","last_clean":"2024-09-22T03:24:31.790279+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:24:31.790279+0000","last_undegraded":"2024-09-22T03:24:31.790279+0000","last_fullsized":"2024-09-22T03:24:31.790279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'864","reported_seq":891,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.788614+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:24:31.788614+0000","last_peered":"2024-09-22T03:24:31.788614+0000","last_clean":"2024-09-22T03:24:31.788614+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:24:31.788614+0000","last_undegraded":"2024-09-22T03:24:31.788614+0000","last_fullsized":"2024-09-22T03:24:31.788614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.719737+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:24:31.719737+0000","last_peered":"2024-09-22T03:24:31.719737+0000","last_clean":"2024-09-22T03:24:31.719737+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:24:31.719737+0000","last_undegraded":"2024-09-22T03:24:31.719737+0000","last_fullsized":"2024-09-22T03:24:31.719737+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'752","reported_seq":779,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.757048+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:24:31.757048+0000","last_peered":"2024-09-22T03:24:31.757048+0000","last_clean":"2024-09-22T03:24:31.757048+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:24:31.757048+0000","last_undegraded":"2024-09-22T03:24:31.757048+0000","last_fullsized":"2024-09-22T03:24:31.757048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.787750+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:31.787750+0000","last_peered":"2024-09-22T03:24:31.787750+0000","last_clean":"2024-09-22T03:24:31.787750+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:24:31.787750+0000","last_undegraded":"2024-09-22T03:24:31.787750+0000","last_fullsized":"2024-09-22T03:24:31.787750+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'928","reported_seq":955,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.759603+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:24:31.759603+0000","last_peered":"2024-09-22T03:24:31.759603+0000","last_clean":"2024-09-22T03:24:31.759603+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:24:31.759603+0000","last_undegraded":"2024-09-22T03:24:31.759603+0000","last_fullsized":"2024-09-22T03:24:31.759603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.717549+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:31.717549+0000","last_peered":"2024-09-22T03:24:31.717549+0000","last_clean":"2024-09-22T03:24:31.717549+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:24:31.717549+0000","last_undegraded":"2024-09-22T03:24:31.717549+0000","last_fullsized":"2024-09-22T03:24:31.717549+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'752","reported_seq":779,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.708086+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:24:31.708086+0000","last_peered":"2024-09-22T03:24:31.708086+0000","last_clean":"2024-09-22T03:24:31.708086+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:24:31.708086+0000","last_undegraded":"2024-09-22T03:24:31.708086+0000","last_fullsized":"2024-09-22T03:24:31.708086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'824","reported_seq":851,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.787400+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:24:31.787400+0000","last_peered":"2024-09-22T03:24:31.787400+0000","last_clean":"2024-09-22T03:24:31.787400+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:24:31.787400+0000","last_undegraded":"2024-09-22T03:24:31.787400+0000","last_fullsized":"2024-09-22T03:24:31.787400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.715269+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:31.715269+0000","last_peered":"2024-09-22T03:24:31.715269+0000","last_clean":"2024-09-22T03:24:31.715269+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:24:31.715269+0000","last_undegraded":"2024-09-22T03:24:31.715269+0000","last_fullsized":"2024-09-22T03:24:31.715269+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'1008","reported_seq":1035,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.713880+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:24:31.713880+0000","last_peered":"2024-09-22T03:24:31.713880+0000","last_clean":"2024-09-22T03:24:31.713880+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:24:31.713880+0000","last_undegraded":"2024-09-22T03:24:31.713880+0000","last_fullsized":"2024-09-22T03:24:31.713880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8257536,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.758102+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:31.758102+0000","last_peered":"2024-09-22T03:24:31.758102+0000","last_clean":"2024-09-22T03:24:31.758102+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:24:31.758102+0000","last_undegraded":"2024-09-22T03:24:31.758102+0000","last_fullsized":"2024-09-22T03:24:31.758102+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'832","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.757532+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:24:31.757532+0000","last_peered":"2024-09-22T03:24:31.757532+0000","last_clean":"2024-09-22T03:24:31.757532+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:24:31.757532+0000","last_undegraded":"2024-09-22T03:24:31.757532+0000","last_fullsized":"2024-09-22T03:24:31.757532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.709778+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:24:31.709778+0000","last_peered":"2024-09-22T03:24:31.709778+0000","last_clean":"2024-09-22T03:24:31.709778+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:24:31.709778+0000","last_undegraded":"2024-09-22T03:24:31.709778+0000","last_fullsized":"2024-09-22T03:24:31.709778+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'999","reported_seq":1027,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.715125+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:24:31.715125+0000","last_peered":"2024-09-22T03:24:31.715125+0000","last_clean":"2024-09-22T03:24:31.715125+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:24:31.715125+0000","last_undegraded":"2024-09-22T03:24:31.715125+0000","last_fullsized":"2024-09-22T03:24:31.715125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":125,"num_object_clones":0,"num_object_copies":375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1000,"num_write_kb":8000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.789010+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:24:31.789010+0000","last_peered":"2024-09-22T03:24:31.789010+0000","last_clean":"2024-09-22T03:24:31.789010+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:24:31.789010+0000","last_undegraded":"2024-09-22T03:24:31.789010+0000","last_fullsized":"2024-09-22T03:24:31.789010+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":112205824,"num_objects":1713,"num_object_clones":0,"num_object_copies":5139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13697,"num_write_kb":109576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213506334720,"internally_reserved":0,"allocated":705159168,"data_stored":705159168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13693,"ondisk_log_size":13693,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213506334720,"internally_reserved":0,"allocated":705159168,"data_stored":705159168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":199736,"kb_used_data":199736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52097992,"statfs":{"total":53552873472,"available":53348343808,"internally_reserved":0,"allocated":204529664,"data_stored":204529664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":160620,"kb_used_data":160620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52137108,"statfs":{"total":53552873472,"available":53388398592,"internally_reserved":0,"allocated":164474880,"data_stored":164474880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":171920,"kb_used_data":171920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125808,"statfs":{"total":53552873472,"available":53376827392,"internally_reserved":0,"allocated":176046080,"data_stored":176046080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":155628,"kb_used_data":155628,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52142100,"statfs":{"total":53552873472,"available":53393510400,"internally_reserved":0,"allocated":159363072,"data_stored":159363072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53393387520,"internally_reserved":0,"allocated":159485952,"data_stored":159485952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53376581632,"internally_reserved":0,"allocated":176291840,"data_stored":176291840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53388152832,"internally_reserved":0,"allocated":164720640,"data_stored":164720640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53393387520,"internally_reserved":0,"allocated":159485952,"data_stored":159485952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53376581632,"internally_reserved":0,"allocated":176291840,"data_stored":176291840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53388152832,"internally_reserved":0,"allocated":164720640,"data_stored":164720640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:35.232 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:35.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:35.345+0000 7f5c7886c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:35.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:35.355+0000 7f5c7886c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:24:35.663 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:24:35.683 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-22T03:24:34.708192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112796211,"num_objects":1717,"num_object_clones":0,"num_object_copies":5151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13836,"num_write_kb":111472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13793,"ondisk_log_size":13793,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":687904,"kb_used_data":687904,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208503008,"statfs":{"total":214211493888,"available":213507080192,"internally_reserved":0,"allocated":704413696,"data_stored":704413696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6791168,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":6632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.756300"},"pg_stats":[{"pgid":"3.f","version":"23'896","reported_seq":923,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.721980+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:24:31.721980+0000","last_peered":"2024-09-22T03:24:31.721980+0000","last_clean":"2024-09-22T03:24:31.721980+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:24:31.721980+0000","last_undegraded":"2024-09-22T03:24:31.721980+0000","last_fullsized":"2024-09-22T03:24:31.721980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'832","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.714342+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:24:31.714342+0000","last_peered":"2024-09-22T03:24:31.714342+0000","last_clean":"2024-09-22T03:24:31.714342+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:24:31.714342+0000","last_undegraded":"2024-09-22T03:24:31.714342+0000","last_fullsized":"2024-09-22T03:24:31.714342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'784","reported_seq":811,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.790943+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:24:31.790943+0000","last_peered":"2024-09-22T03:24:31.790943+0000","last_clean":"2024-09-22T03:24:31.790943+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:24:31.790943+0000","last_undegraded":"2024-09-22T03:24:31.790943+0000","last_fullsized":"2024-09-22T03:24:31.790943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'800","reported_seq":827,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.719021+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:24:31.719021+0000","last_peered":"2024-09-22T03:24:31.719021+0000","last_clean":"2024-09-22T03:24:31.719021+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:24:31.719021+0000","last_undegraded":"2024-09-22T03:24:31.719021+0000","last_fullsized":"2024-09-22T03:24:31.719021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'831","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.910931+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:24:31.910931+0000","last_peered":"2024-09-22T03:24:31.910931+0000","last_clean":"2024-09-22T03:24:31.910931+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:24:31.910931+0000","last_undegraded":"2024-09-22T03:24:31.910931+0000","last_fullsized":"2024-09-22T03:24:31.910931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'872","reported_seq":899,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.716540+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:24:31.716540+0000","last_peered":"2024-09-22T03:24:31.716540+0000","last_clean":"2024-09-22T03:24:31.716540+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:24:31.716540+0000","last_undegraded":"2024-09-22T03:24:31.716540+0000","last_fullsized":"2024-09-22T03:24:31.716540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"24'880","reported_seq":908,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.858407+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:24:31.858407+0000","last_peered":"2024-09-22T03:24:31.858407+0000","last_clean":"2024-09-22T03:24:31.858407+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:24:31.858407+0000","last_undegraded":"2024-09-22T03:24:31.858407+0000","last_fullsized":"2024-09-22T03:24:31.858407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7217152,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":881,"num_write_kb":7048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'839","reported_seq":867,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.789216+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:24:31.789216+0000","last_peered":"2024-09-22T03:24:31.789216+0000","last_clean":"2024-09-22T03:24:31.789216+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:24:31.789216+0000","last_undegraded":"2024-09-22T03:24:31.789216+0000","last_fullsized":"2024-09-22T03:24:31.789216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881280,"num_objects":105,"num_object_clones":0,"num_object_copies":315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.755710+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:24:31.755710+0000","last_peered":"2024-09-22T03:24:31.755710+0000","last_clean":"2024-09-22T03:24:31.755710+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:24:31.755710+0000","last_undegraded":"2024-09-22T03:24:31.755710+0000","last_fullsized":"2024-09-22T03:24:31.755710+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":41,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.790279+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:24:31.790279+0000","last_peered":"2024-09-22T03:24:31.790279+0000","last_clean":"2024-09-22T03:24:31.790279+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:24:31.790279+0000","last_undegraded":"2024-09-22T03:24:31.790279+0000","last_fullsized":"2024-09-22T03:24:31.790279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'864","reported_seq":891,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.788614+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:24:31.788614+0000","last_peered":"2024-09-22T03:24:31.788614+0000","last_clean":"2024-09-22T03:24:31.788614+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:24:31.788614+0000","last_undegraded":"2024-09-22T03:24:31.788614+0000","last_fullsized":"2024-09-22T03:24:31.788614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.719737+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:24:31.719737+0000","last_peered":"2024-09-22T03:24:31.719737+0000","last_clean":"2024-09-22T03:24:31.719737+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:24:31.719737+0000","last_undegraded":"2024-09-22T03:24:31.719737+0000","last_fullsized":"2024-09-22T03:24:31.719737+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'752","reported_seq":779,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.757048+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:24:31.757048+0000","last_peered":"2024-09-22T03:24:31.757048+0000","last_clean":"2024-09-22T03:24:31.757048+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:24:31.757048+0000","last_undegraded":"2024-09-22T03:24:31.757048+0000","last_fullsized":"2024-09-22T03:24:31.757048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.787750+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:31.787750+0000","last_peered":"2024-09-22T03:24:31.787750+0000","last_clean":"2024-09-22T03:24:31.787750+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:24:31.787750+0000","last_undegraded":"2024-09-22T03:24:31.787750+0000","last_fullsized":"2024-09-22T03:24:31.787750+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'928","reported_seq":955,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.759603+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:24:31.759603+0000","last_peered":"2024-09-22T03:24:31.759603+0000","last_clean":"2024-09-22T03:24:31.759603+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:24:31.759603+0000","last_undegraded":"2024-09-22T03:24:31.759603+0000","last_fullsized":"2024-09-22T03:24:31.759603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.717549+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:31.717549+0000","last_peered":"2024-09-22T03:24:31.717549+0000","last_clean":"2024-09-22T03:24:31.717549+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:24:31.717549+0000","last_undegraded":"2024-09-22T03:24:31.717549+0000","last_fullsized":"2024-09-22T03:24:31.717549+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'752","reported_seq":779,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.708086+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:24:31.708086+0000","last_peered":"2024-09-22T03:24:31.708086+0000","last_clean":"2024-09-22T03:24:31.708086+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:24:31.708086+0000","last_undegraded":"2024-09-22T03:24:31.708086+0000","last_fullsized":"2024-09-22T03:24:31.708086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'824","reported_seq":851,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.787400+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:24:31.787400+0000","last_peered":"2024-09-22T03:24:31.787400+0000","last_clean":"2024-09-22T03:24:31.787400+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:24:31.787400+0000","last_undegraded":"2024-09-22T03:24:31.787400+0000","last_fullsized":"2024-09-22T03:24:31.787400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.715269+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:31.715269+0000","last_peered":"2024-09-22T03:24:31.715269+0000","last_clean":"2024-09-22T03:24:31.715269+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:24:31.715269+0000","last_undegraded":"2024-09-22T03:24:31.715269+0000","last_fullsized":"2024-09-22T03:24:31.715269+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'1008","reported_seq":1035,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.713880+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:24:31.713880+0000","last_peered":"2024-09-22T03:24:31.713880+0000","last_clean":"2024-09-22T03:24:31.713880+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:24:31.713880+0000","last_undegraded":"2024-09-22T03:24:31.713880+0000","last_fullsized":"2024-09-22T03:24:31.713880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8257536,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.758102+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:31.758102+0000","last_peered":"2024-09-22T03:24:31.758102+0000","last_clean":"2024-09-22T03:24:31.758102+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:24:31.758102+0000","last_undegraded":"2024-09-22T03:24:31.758102+0000","last_fullsized":"2024-09-22T03:24:31.758102+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'832","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.757532+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:24:31.757532+0000","last_peered":"2024-09-22T03:24:31.757532+0000","last_clean":"2024-09-22T03:24:31.757532+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:24:31.757532+0000","last_undegraded":"2024-09-22T03:24:31.757532+0000","last_fullsized":"2024-09-22T03:24:31.757532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.709778+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:24:31.709778+0000","last_peered":"2024-09-22T03:24:31.709778+0000","last_clean":"2024-09-22T03:24:31.709778+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:24:31.709778+0000","last_undegraded":"2024-09-22T03:24:31.709778+0000","last_fullsized":"2024-09-22T03:24:31.709778+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'999","reported_seq":1027,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.715125+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:24:31.715125+0000","last_peered":"2024-09-22T03:24:31.715125+0000","last_clean":"2024-09-22T03:24:31.715125+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:24:31.715125+0000","last_undegraded":"2024-09-22T03:24:31.715125+0000","last_fullsized":"2024-09-22T03:24:31.715125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":125,"num_object_clones":0,"num_object_copies":375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1000,"num_write_kb":8000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.789010+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:24:31.789010+0000","last_peered":"2024-09-22T03:24:31.789010+0000","last_clean":"2024-09-22T03:24:31.789010+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:24:31.789010+0000","last_undegraded":"2024-09-22T03:24:31.789010+0000","last_fullsized":"2024-09-22T03:24:31.789010+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":112205824,"num_objects":1713,"num_object_clones":0,"num_object_copies":5139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13697,"num_write_kb":109576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213506334720,"internally_reserved":0,"allocated":705159168,"data_stored":705159168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13693,"ondisk_log_size":13693,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213506334720,"internally_reserved":0,"allocated":705159168,"data_stored":705159168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":199736,"kb_used_data":199736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52097992,"statfs":{"total":53552873472,"available":53348343808,"internally_reserved":0,"allocated":204529664,"data_stored":204529664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":160620,"kb_used_data":160620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52137108,"statfs":{"total":53552873472,"available":53388398592,"internally_reserved":0,"allocated":164474880,"data_stored":164474880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":171920,"kb_used_data":171920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125808,"statfs":{"total":53552873472,"available":53376827392,"internally_reserved":0,"allocated":176046080,"data_stored":176046080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":155628,"kb_used_data":155628,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52142100,"statfs":{"total":53552873472,"available":53393510400,"internally_reserved":0,"allocated":159363072,"data_stored":159363072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53393387520,"internally_reserved":0,"allocated":159485952,"data_stored":159485952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53376581632,"internally_reserved":0,"allocated":176291840,"data_stored":176291840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53388152832,"internally_reserved":0,"allocated":164720640,"data_stored":164720640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53393387520,"internally_reserved":0,"allocated":159485952,"data_stored":159485952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53376581632,"internally_reserved":0,"allocated":176291840,"data_stored":176291840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53388152832,"internally_reserved":0,"allocated":164720640,"data_stored":164720640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:35.684 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:24:35.685 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:35.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:35.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:35.803+0000 7f3940cd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:35.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:35.814+0000 7f3940cd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:36.035 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:24:36.036 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:24:36.053 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-22T03:24:35.953212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112796211,"num_objects":1717,"num_object_clones":0,"num_object_copies":5151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13836,"num_write_kb":111472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13793,"ondisk_log_size":13793,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":687904,"kb_used_data":687904,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208503008,"statfs":{"total":214211493888,"available":213507080192,"internally_reserved":0,"allocated":704413696,"data_stored":704413696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6791168,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":6632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.260196"},"pg_stats":[{"pgid":"3.f","version":"23'896","reported_seq":923,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.721980+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:24:31.721980+0000","last_peered":"2024-09-22T03:24:31.721980+0000","last_clean":"2024-09-22T03:24:31.721980+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:24:31.721980+0000","last_undegraded":"2024-09-22T03:24:31.721980+0000","last_fullsized":"2024-09-22T03:24:31.721980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"23'832","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.714342+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:24:31.714342+0000","last_peered":"2024-09-22T03:24:31.714342+0000","last_clean":"2024-09-22T03:24:31.714342+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:24:31.714342+0000","last_undegraded":"2024-09-22T03:24:31.714342+0000","last_fullsized":"2024-09-22T03:24:31.714342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'784","reported_seq":811,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.790943+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:24:31.790943+0000","last_peered":"2024-09-22T03:24:31.790943+0000","last_clean":"2024-09-22T03:24:31.790943+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:24:31.790943+0000","last_undegraded":"2024-09-22T03:24:31.790943+0000","last_fullsized":"2024-09-22T03:24:31.790943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'800","reported_seq":827,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.719021+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:24:31.719021+0000","last_peered":"2024-09-22T03:24:31.719021+0000","last_clean":"2024-09-22T03:24:31.719021+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:24:31.719021+0000","last_undegraded":"2024-09-22T03:24:31.719021+0000","last_fullsized":"2024-09-22T03:24:31.719021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'831","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.910931+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:24:31.910931+0000","last_peered":"2024-09-22T03:24:31.910931+0000","last_clean":"2024-09-22T03:24:31.910931+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:24:31.910931+0000","last_undegraded":"2024-09-22T03:24:31.910931+0000","last_fullsized":"2024-09-22T03:24:31.910931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"23'872","reported_seq":899,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.716540+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:24:31.716540+0000","last_peered":"2024-09-22T03:24:31.716540+0000","last_clean":"2024-09-22T03:24:31.716540+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:24:31.716540+0000","last_undegraded":"2024-09-22T03:24:31.716540+0000","last_fullsized":"2024-09-22T03:24:31.716540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"24'880","reported_seq":908,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.858407+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:24:31.858407+0000","last_peered":"2024-09-22T03:24:31.858407+0000","last_clean":"2024-09-22T03:24:31.858407+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:24:31.858407+0000","last_undegraded":"2024-09-22T03:24:31.858407+0000","last_fullsized":"2024-09-22T03:24:31.858407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7217152,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":881,"num_write_kb":7048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"23'839","reported_seq":867,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.789216+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:24:31.789216+0000","last_peered":"2024-09-22T03:24:31.789216+0000","last_clean":"2024-09-22T03:24:31.789216+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:24:31.789216+0000","last_undegraded":"2024-09-22T03:24:31.789216+0000","last_fullsized":"2024-09-22T03:24:31.789216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881280,"num_objects":105,"num_object_clones":0,"num_object_copies":315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.755710+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:24:31.755710+0000","last_peered":"2024-09-22T03:24:31.755710+0000","last_clean":"2024-09-22T03:24:31.755710+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:24:31.755710+0000","last_undegraded":"2024-09-22T03:24:31.755710+0000","last_fullsized":"2024-09-22T03:24:31.755710+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":41,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.790279+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:24:31.790279+0000","last_peered":"2024-09-22T03:24:31.790279+0000","last_clean":"2024-09-22T03:24:31.790279+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:24:31.790279+0000","last_undegraded":"2024-09-22T03:24:31.790279+0000","last_fullsized":"2024-09-22T03:24:31.790279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"23'864","reported_seq":891,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.788614+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:24:31.788614+0000","last_peered":"2024-09-22T03:24:31.788614+0000","last_clean":"2024-09-22T03:24:31.788614+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:24:31.788614+0000","last_undegraded":"2024-09-22T03:24:31.788614+0000","last_fullsized":"2024-09-22T03:24:31.788614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.719737+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:24:31.719737+0000","last_peered":"2024-09-22T03:24:31.719737+0000","last_clean":"2024-09-22T03:24:31.719737+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:24:31.719737+0000","last_undegraded":"2024-09-22T03:24:31.719737+0000","last_fullsized":"2024-09-22T03:24:31.719737+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"23'752","reported_seq":779,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.757048+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:24:31.757048+0000","last_peered":"2024-09-22T03:24:31.757048+0000","last_clean":"2024-09-22T03:24:31.757048+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:24:31.757048+0000","last_undegraded":"2024-09-22T03:24:31.757048+0000","last_fullsized":"2024-09-22T03:24:31.757048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.787750+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:31.787750+0000","last_peered":"2024-09-22T03:24:31.787750+0000","last_clean":"2024-09-22T03:24:31.787750+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:24:31.787750+0000","last_undegraded":"2024-09-22T03:24:31.787750+0000","last_fullsized":"2024-09-22T03:24:31.787750+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"23'928","reported_seq":955,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.759603+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:24:31.759603+0000","last_peered":"2024-09-22T03:24:31.759603+0000","last_clean":"2024-09-22T03:24:31.759603+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:24:31.759603+0000","last_undegraded":"2024-09-22T03:24:31.759603+0000","last_fullsized":"2024-09-22T03:24:31.759603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":42,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.717549+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:31.717549+0000","last_peered":"2024-09-22T03:24:31.717549+0000","last_clean":"2024-09-22T03:24:31.717549+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:24:31.717549+0000","last_undegraded":"2024-09-22T03:24:31.717549+0000","last_fullsized":"2024-09-22T03:24:31.717549+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"23'752","reported_seq":779,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.708086+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:24:31.708086+0000","last_peered":"2024-09-22T03:24:31.708086+0000","last_clean":"2024-09-22T03:24:31.708086+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:24:31.708086+0000","last_undegraded":"2024-09-22T03:24:31.708086+0000","last_fullsized":"2024-09-22T03:24:31.708086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"23'824","reported_seq":851,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.787400+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:24:31.787400+0000","last_peered":"2024-09-22T03:24:31.787400+0000","last_clean":"2024-09-22T03:24:31.787400+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:24:31.787400+0000","last_undegraded":"2024-09-22T03:24:31.787400+0000","last_fullsized":"2024-09-22T03:24:31.787400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.715269+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:31.715269+0000","last_peered":"2024-09-22T03:24:31.715269+0000","last_clean":"2024-09-22T03:24:31.715269+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:24:31.715269+0000","last_undegraded":"2024-09-22T03:24:31.715269+0000","last_fullsized":"2024-09-22T03:24:31.715269+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"23'1008","reported_seq":1035,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.713880+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:24:31.713880+0000","last_peered":"2024-09-22T03:24:31.713880+0000","last_clean":"2024-09-22T03:24:31.713880+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:24:31.713880+0000","last_undegraded":"2024-09-22T03:24:31.713880+0000","last_fullsized":"2024-09-22T03:24:31.713880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8257536,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.758102+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:31.758102+0000","last_peered":"2024-09-22T03:24:31.758102+0000","last_clean":"2024-09-22T03:24:31.758102+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:24:31.758102+0000","last_undegraded":"2024-09-22T03:24:31.758102+0000","last_fullsized":"2024-09-22T03:24:31.758102+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"23'832","reported_seq":859,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.757532+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:24:31.757532+0000","last_peered":"2024-09-22T03:24:31.757532+0000","last_clean":"2024-09-22T03:24:31.757532+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:24:31.757532+0000","last_undegraded":"2024-09-22T03:24:31.757532+0000","last_fullsized":"2024-09-22T03:24:31.757532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.709778+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:24:31.709778+0000","last_peered":"2024-09-22T03:24:31.709778+0000","last_clean":"2024-09-22T03:24:31.709778+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:24:31.709778+0000","last_undegraded":"2024-09-22T03:24:31.709778+0000","last_fullsized":"2024-09-22T03:24:31.709778+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'999","reported_seq":1027,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.715125+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:24:31.715125+0000","last_peered":"2024-09-22T03:24:31.715125+0000","last_clean":"2024-09-22T03:24:31.715125+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:24:31.715125+0000","last_undegraded":"2024-09-22T03:24:31.715125+0000","last_fullsized":"2024-09-22T03:24:31.715125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":125,"num_object_clones":0,"num_object_copies":375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1000,"num_write_kb":8000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-22T03:24:31.789010+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:24:31.789010+0000","last_peered":"2024-09-22T03:24:31.789010+0000","last_clean":"2024-09-22T03:24:31.789010+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:24:31.789010+0000","last_undegraded":"2024-09-22T03:24:31.789010+0000","last_fullsized":"2024-09-22T03:24:31.789010+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":112205824,"num_objects":1713,"num_object_clones":0,"num_object_copies":5139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13697,"num_write_kb":109576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213506334720,"internally_reserved":0,"allocated":705159168,"data_stored":705159168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13693,"ondisk_log_size":13693,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213506334720,"internally_reserved":0,"allocated":705159168,"data_stored":705159168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":199736,"kb_used_data":199736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52097992,"statfs":{"total":53552873472,"available":53348343808,"internally_reserved":0,"allocated":204529664,"data_stored":204529664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":160620,"kb_used_data":160620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52137108,"statfs":{"total":53552873472,"available":53388398592,"internally_reserved":0,"allocated":164474880,"data_stored":164474880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":171920,"kb_used_data":171920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125808,"statfs":{"total":53552873472,"available":53376827392,"internally_reserved":0,"allocated":176046080,"data_stored":176046080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":155628,"kb_used_data":155628,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52142100,"statfs":{"total":53552873472,"available":53393510400,"internally_reserved":0,"allocated":159363072,"data_stored":159363072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53393387520,"internally_reserved":0,"allocated":159485952,"data_stored":159485952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53376581632,"internally_reserved":0,"allocated":176291840,"data_stored":176291840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53388152832,"internally_reserved":0,"allocated":164720640,"data_stored":164720640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53393387520,"internally_reserved":0,"allocated":159485952,"data_stored":159485952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53376581632,"internally_reserved":0,"allocated":176291840,"data_stored":176291840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53388152832,"internally_reserved":0,"allocated":164720640,"data_stored":164720640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53348212736,"internally_reserved":0,"allocated":204660736,"data_stored":204660736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:36.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 14360 14344 1.41839 1.42969 0.079818 0.0880203 2024-09-22T03:24:36.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:36.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:36.281+0000 7fd735597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:36.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:36.287+0000 7fd735597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:37.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:24:37.052648+0000 min lat: 0.0101625 max lat: 0.3442 avg lat: 0.0882591 2024-09-22T03:24:37.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:24:37.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 14508 14492 1.41511 1.15625 0.120894 0.0882591 2024-09-22T03:24:38.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 14693 14677 1.41548 1.44531 0.106592 0.0882361 2024-09-22T03:24:38.417 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:38.538+0000 7f4c8e997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:38.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:38.544+0000 7f4c8e997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:39.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 14845 14829 1.4127 1.1875 0.0383021 0.0884217 2024-09-22T03:24:40.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 15025 15009 1.41262 1.40625 0.0993317 0.0884029 2024-09-22T03:24:40.657 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:40.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:40.791+0000 7f7ef0c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:40.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:40.796+0000 7f7ef0c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:41.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 15185 15169 1.41068 1.25 0.0597206 0.0885133 2024-09-22T03:24:41.056 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:41.056 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:41.056 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:41.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:41.170+0000 7f0491eb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:41.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:41.179+0000 7f0491eb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:41.474 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:24:41.475 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:24:41.489 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-22T03:24:39.953912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":119317043,"num_objects":1816,"num_object_clones":0,"num_object_copies":5448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":14632,"num_write_kb":117840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14577,"ondisk_log_size":14577,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":727192,"kb_used_data":727192,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208463720,"statfs":{"total":214211493888,"available":213466849280,"internally_reserved":0,"allocated":744644608,"data_stored":744644608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6520832,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":796,"num_write_kb":6368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.256575"},"pg_stats":[{"pgid":"3.f","version":"27'935","reported_seq":972,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:35.796368+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:24:35.796368+0000","last_peered":"2024-09-22T03:24:35.796368+0000","last_clean":"2024-09-22T03:24:35.796368+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:24:35.796368+0000","last_undegraded":"2024-09-22T03:24:35.796368+0000","last_fullsized":"2024-09-22T03:24:35.796368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"27'836","reported_seq":873,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.943365+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:24:36.943365+0000","last_peered":"2024-09-22T03:24:36.943365+0000","last_clean":"2024-09-22T03:24:36.943365+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:24:36.943365+0000","last_undegraded":"2024-09-22T03:24:36.943365+0000","last_fullsized":"2024-09-22T03:24:36.943365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":105,"num_object_clones":0,"num_object_copies":315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":837,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"27'815","reported_seq":852,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.415145+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:24:36.415145+0000","last_peered":"2024-09-22T03:24:36.415145+0000","last_clean":"2024-09-22T03:24:36.415145+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:24:36.415145+0000","last_undegraded":"2024-09-22T03:24:36.415145+0000","last_fullsized":"2024-09-22T03:24:36.415145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'879","reported_seq":916,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.906821+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:24:36.906821+0000","last_peered":"2024-09-22T03:24:36.906821+0000","last_clean":"2024-09-22T03:24:36.906821+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:24:36.906821+0000","last_undegraded":"2024-09-22T03:24:36.906821+0000","last_fullsized":"2024-09-22T03:24:36.906821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"27'863","reported_seq":900,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.460688+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:24:36.460688+0000","last_peered":"2024-09-22T03:24:36.460688+0000","last_clean":"2024-09-22T03:24:36.460688+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:24:36.460688+0000","last_undegraded":"2024-09-22T03:24:36.460688+0000","last_fullsized":"2024-09-22T03:24:36.460688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"27'927","reported_seq":964,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.180943+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:24:36.180943+0000","last_peered":"2024-09-22T03:24:36.180943+0000","last_clean":"2024-09-22T03:24:36.180943+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:24:36.180943+0000","last_undegraded":"2024-09-22T03:24:36.180943+0000","last_fullsized":"2024-09-22T03:24:36.180943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"27'927","reported_seq":964,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.627923+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:24:36.627923+0000","last_peered":"2024-09-22T03:24:36.627923+0000","last_clean":"2024-09-22T03:24:36.627923+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:24:36.627923+0000","last_undegraded":"2024-09-22T03:24:36.627923+0000","last_fullsized":"2024-09-22T03:24:36.627923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'895","reported_seq":932,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:35.415108+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:24:35.415108+0000","last_peered":"2024-09-22T03:24:35.415108+0000","last_clean":"2024-09-22T03:24:35.415108+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:24:35.415108+0000","last_undegraded":"2024-09-22T03:24:35.415108+0000","last_fullsized":"2024-09-22T03:24:35.415108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.735395+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:24:34.735395+0000","last_peered":"2024-09-22T03:24:34.735395+0000","last_clean":"2024-09-22T03:24:34.735395+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:24:34.735395+0000","last_undegraded":"2024-09-22T03:24:34.735395+0000","last_fullsized":"2024-09-22T03:24:34.735395+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.729094+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:24:34.729094+0000","last_peered":"2024-09-22T03:24:34.729094+0000","last_clean":"2024-09-22T03:24:34.729094+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:24:34.729094+0000","last_undegraded":"2024-09-22T03:24:34.729094+0000","last_fullsized":"2024-09-22T03:24:34.729094+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"27'951","reported_seq":988,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.299207+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:24:36.299207+0000","last_peered":"2024-09-22T03:24:36.299207+0000","last_clean":"2024-09-22T03:24:36.299207+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:24:36.299207+0000","last_undegraded":"2024-09-22T03:24:36.299207+0000","last_fullsized":"2024-09-22T03:24:36.299207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.745191+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:24:34.745191+0000","last_peered":"2024-09-22T03:24:34.745191+0000","last_clean":"2024-09-22T03:24:34.745191+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:24:34.745191+0000","last_undegraded":"2024-09-22T03:24:34.745191+0000","last_fullsized":"2024-09-22T03:24:34.745191+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"27'799","reported_seq":836,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.783794+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:24:36.783794+0000","last_peered":"2024-09-22T03:24:36.783794+0000","last_clean":"2024-09-22T03:24:36.783794+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:24:36.783794+0000","last_undegraded":"2024-09-22T03:24:36.783794+0000","last_fullsized":"2024-09-22T03:24:36.783794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.724879+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:34.724879+0000","last_peered":"2024-09-22T03:24:34.724879+0000","last_clean":"2024-09-22T03:24:34.724879+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:24:34.724879+0000","last_undegraded":"2024-09-22T03:24:34.724879+0000","last_fullsized":"2024-09-22T03:24:34.724879+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'991","reported_seq":1028,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.170288+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:24:36.170288+0000","last_peered":"2024-09-22T03:24:36.170288+0000","last_clean":"2024-09-22T03:24:36.170288+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:24:36.170288+0000","last_undegraded":"2024-09-22T03:24:36.170288+0000","last_fullsized":"2024-09-22T03:24:36.170288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.740175+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:34.740175+0000","last_peered":"2024-09-22T03:24:34.740175+0000","last_clean":"2024-09-22T03:24:34.740175+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:24:34.740175+0000","last_undegraded":"2024-09-22T03:24:34.740175+0000","last_fullsized":"2024-09-22T03:24:34.740175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"27'791","reported_seq":828,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:35.851401+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:24:35.851401+0000","last_peered":"2024-09-22T03:24:35.851401+0000","last_clean":"2024-09-22T03:24:35.851401+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:24:35.851401+0000","last_undegraded":"2024-09-22T03:24:35.851401+0000","last_fullsized":"2024-09-22T03:24:35.851401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"27'871","reported_seq":908,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:35.498928+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:24:35.498928+0000","last_peered":"2024-09-22T03:24:35.498928+0000","last_clean":"2024-09-22T03:24:35.498928+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:24:35.498928+0000","last_undegraded":"2024-09-22T03:24:35.498928+0000","last_fullsized":"2024-09-22T03:24:35.498928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.720641+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:34.720641+0000","last_peered":"2024-09-22T03:24:34.720641+0000","last_clean":"2024-09-22T03:24:34.720641+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:24:34.720641+0000","last_undegraded":"2024-09-22T03:24:34.720641+0000","last_fullsized":"2024-09-22T03:24:34.720641+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"27'1039","reported_seq":1076,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.561593+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:24:36.561593+0000","last_peered":"2024-09-22T03:24:36.561593+0000","last_clean":"2024-09-22T03:24:36.561593+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:24:36.561593+0000","last_undegraded":"2024-09-22T03:24:36.561593+0000","last_fullsized":"2024-09-22T03:24:36.561593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1039,"log_dups_size":0,"ondisk_log_size":1039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8519680,"num_objects":130,"num_object_clones":0,"num_object_copies":390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1040,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.745913+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:34.745913+0000","last_peered":"2024-09-22T03:24:34.745913+0000","last_clean":"2024-09-22T03:24:34.745913+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:24:34.745913+0000","last_undegraded":"2024-09-22T03:24:34.745913+0000","last_fullsized":"2024-09-22T03:24:34.745913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'895","reported_seq":932,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.708015+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:24:36.708015+0000","last_peered":"2024-09-22T03:24:36.708015+0000","last_clean":"2024-09-22T03:24:36.708015+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:24:36.708015+0000","last_undegraded":"2024-09-22T03:24:36.708015+0000","last_fullsized":"2024-09-22T03:24:36.708015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.720872+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:24:34.720872+0000","last_peered":"2024-09-22T03:24:34.720872+0000","last_clean":"2024-09-22T03:24:34.720872+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:24:34.720872+0000","last_undegraded":"2024-09-22T03:24:34.720872+0000","last_fullsized":"2024-09-22T03:24:34.720872+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'1063","reported_seq":1100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:36.846835+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:24:36.846835+0000","last_peered":"2024-09-22T03:24:36.846835+0000","last_clean":"2024-09-22T03:24:36.846835+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:24:36.846835+0000","last_undegraded":"2024-09-22T03:24:36.846835+0000","last_fullsized":"2024-09-22T03:24:36.846835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":133,"num_object_clones":0,"num_object_copies":399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1064,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.728445+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:24:34.728445+0000","last_peered":"2024-09-22T03:24:34.728445+0000","last_clean":"2024-09-22T03:24:34.728445+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:24:34.728445+0000","last_undegraded":"2024-09-22T03:24:34.728445+0000","last_fullsized":"2024-09-22T03:24:34.728445+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":118726656,"num_objects":1812,"num_object_clones":0,"num_object_copies":5436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14493,"num_write_kb":115944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213466091520,"internally_reserved":0,"allocated":745402368,"data_stored":745402368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14477,"ondisk_log_size":14477,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213466091520,"internally_reserved":0,"allocated":745402368,"data_stored":745402368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53336600576,"internally_reserved":0,"allocated":216272896,"data_stored":216272896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":210960,"kb_used_data":210960,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52086768,"statfs":{"total":53552873472,"available":53336850432,"internally_reserved":0,"allocated":216023040,"data_stored":216023040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":169236,"kb_used_data":169236,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52128492,"statfs":{"total":53552873472,"available":53379575808,"internally_reserved":0,"allocated":173297664,"data_stored":173297664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":182048,"kb_used_data":182048,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52115680,"statfs":{"total":53552873472,"available":53366456320,"internally_reserved":0,"allocated":186417152,"data_stored":186417152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":164948,"kb_used_data":164948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52132780,"statfs":{"total":53552873472,"available":53383966720,"internally_reserved":0,"allocated":168906752,"data_stored":168906752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53336600576,"internally_reserved":0,"allocated":216272896,"data_stored":216272896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53383835648,"internally_reserved":0,"allocated":169037824,"data_stored":169037824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53366333440,"internally_reserved":0,"allocated":186540032,"data_stored":186540032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53379321856,"internally_reserved":0,"allocated":173551616,"data_stored":173551616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53336600576,"internally_reserved":0,"allocated":216272896,"data_stored":216272896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53383835648,"internally_reserved":0,"allocated":169037824,"data_stored":169037824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53366333440,"internally_reserved":0,"allocated":186540032,"data_stored":186540032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53379321856,"internally_reserved":0,"allocated":173551616,"data_stored":173551616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53336600576,"internally_reserved":0,"allocated":216272896,"data_stored":216272896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:42.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 15367 15351 1.41081 1.42188 0.118728 0.0884912 2024-09-22T03:24:42.934 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:43.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 15517 15501 1.40803 1.17188 0.103214 0.0886985 2024-09-22T03:24:43.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:43.051+0000 7f0c66f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:43.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:43.057+0000 7f0c66f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:44.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 15670 15654 1.40559 1.19531 0.129278 0.0888405 2024-09-22T03:24:45.053 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 15817 15801 1.40266 1.14844 0.0496469 0.0890671 2024-09-22T03:24:45.187 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:45.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:45.307+0000 7f8df470e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:45.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:45.312+0000 7f8df470e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:46.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 15994 15978 1.40244 1.38281 0.173862 0.0890617 2024-09-22T03:24:46.491 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:46.492 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:46.492 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:24:46.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:24:46.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:46.612+0000 7fdddb823640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:46.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:46.621+0000 7fdddb823640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:46.649 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:46.649 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:46.649 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:46.649 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:46.649 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:46.649 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:46.649 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:46.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:46.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:46.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:46.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:46.650 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:46.652 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:46.653 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:46.653 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:24:46.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:46.766+0000 7f9697317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:46.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:46.775+0000 7f9697317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:46.790 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:46.791 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:46.791 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:46.791 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:46.791 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:46.791 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:46.791 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:46.794 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:46.794 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:46.795 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:24:46.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:46.927+0000 7f6aa3597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:46.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:46.939+0000 7f6aa3597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:46.983 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:46.983 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:46.983 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:46.984 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:46.987 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:46.987 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:46.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:24:47.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 15 16168 16153 1.40205 1.36719 0.100342 0.0891009 2024-09-22T03:24:47.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:47.099+0000 7f2e0db97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:47.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:47.108+0000 7f2e0db97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:47.124 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:47.127 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:47.127 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:47.453 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:47.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:47.551+0000 7f5342458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:47.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:47.556+0000 7f5342458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:48.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.0763 2024-09-22T03:24:48.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 16168 2024-09-22T03:24:48.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:24:48.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:24:48.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.40228 2024-09-22T03:24:48.054 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.138201 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.71875 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 1.125 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 179 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 17.6897 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 220 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 144 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.0890974 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.0482219 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.3442 2024-09-22T03:24:48.055 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0101625 2024-09-22T03:24:48.074 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:24:49.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:49.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:49.770+0000 7f22fe46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:49.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:49.777+0000 7f22fe46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:51.972 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:52.128 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:52.128 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:52.128 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:24:52.128 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:24:52.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.126+0000 7f44cc2d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.133+0000 7f44cc2d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.273+0000 7fef92f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.287+0000 7fef92f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.316 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:52.316 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:52.316 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:52.316 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:52.316 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:52.317 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:52.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:52.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:52.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:52.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:52.317 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:52.317 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:52.321 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:52.321 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:52.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:24:52.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.496+0000 7f304706c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.506+0000 7f304706c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.532 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:52.532 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:52.533 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:52.536 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:52.537 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:52.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:24:52.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.670+0000 7f7fe326c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.684+0000 7f7fe326c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.732 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:52.733 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:52.736 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:52.736 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:52.737 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:24:52.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.886+0000 7f7d80797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:52.897+0000 7f7d80797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:52.959 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:24:52.959 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:24:52.960 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:24:52.963 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:24:52.963 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:24:54.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:54.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:54.427+0000 7ff548757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:54.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:54.435+0000 7ff548757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:56.650 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:24:56.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:56.786+0000 7fc7f5317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:56.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:56.793+0000 7fc7f5317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:57.964 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:24:57.965 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:24:57.965 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:24:58.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:58.092+0000 7f2bd2157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:58.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:58.105+0000 7f2bd2157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:58.343 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:24:58.343 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:24:58.366 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-22T03:24:57.957026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19989067,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":18033,"num_write_kb":131241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17978,"ondisk_log_size":17978,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":158912,"kb_used_data":158912,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209032000,"statfs":{"total":214211493888,"available":214048768000,"internally_reserved":0,"allocated":162725888,"data_stored":162725888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-106315752,"num_objects":-1623,"num_object_clones":0,"num_object_copies":-4869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2548,"num_write_kb":6577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002107"},"pg_stats":[{"pgid":"3.f","version":"27'1175","reported_seq":1212,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.819210+0000","last_change":"2024-09-22T03:23:16.082020+0000","last_active":"2024-09-22T03:24:56.819210+0000","last_peered":"2024-09-22T03:24:56.819210+0000","last_clean":"2024-09-22T03:24:56.819210+0000","last_became_active":"2024-09-22T03:23:16.074597+0000","last_became_peered":"2024-09-22T03:23:16.074597+0000","last_unstale":"2024-09-22T03:24:56.819210+0000","last_undegraded":"2024-09-22T03:24:56.819210+0000","last_fullsized":"2024-09-22T03:24:56.819210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1175,"log_dups_size":0,"ondisk_log_size":1175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"27'1087","reported_seq":1124,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.562834+0000","last_change":"2024-09-22T03:23:16.071510+0000","last_active":"2024-09-22T03:24:56.562834+0000","last_peered":"2024-09-22T03:24:56.562834+0000","last_clean":"2024-09-22T03:24:56.562834+0000","last_became_active":"2024-09-22T03:23:16.064813+0000","last_became_peered":"2024-09-22T03:23:16.064813+0000","last_unstale":"2024-09-22T03:24:56.562834+0000","last_undegraded":"2024-09-22T03:24:56.562834+0000","last_fullsized":"2024-09-22T03:24:56.562834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"27'1018","reported_seq":1056,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.703742+0000","last_change":"2024-09-22T03:23:16.067551+0000","last_active":"2024-09-22T03:24:56.703742+0000","last_peered":"2024-09-22T03:24:56.703742+0000","last_clean":"2024-09-22T03:24:56.703742+0000","last_became_active":"2024-09-22T03:23:16.060125+0000","last_became_peered":"2024-09-22T03:23:16.060125+0000","last_unstale":"2024-09-22T03:24:56.703742+0000","last_undegraded":"2024-09-22T03:24:56.703742+0000","last_fullsized":"2024-09-22T03:24:56.703742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1018,"log_dups_size":0,"ondisk_log_size":1018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114136,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1019,"num_write_kb":7361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'1060","reported_seq":1097,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.662965+0000","last_change":"2024-09-22T03:23:16.069219+0000","last_active":"2024-09-22T03:24:56.662965+0000","last_peered":"2024-09-22T03:24:56.662965+0000","last_clean":"2024-09-22T03:24:56.662965+0000","last_became_active":"2024-09-22T03:23:16.054527+0000","last_became_peered":"2024-09-22T03:23:16.054527+0000","last_unstale":"2024-09-22T03:24:56.662965+0000","last_undegraded":"2024-09-22T03:24:56.662965+0000","last_fullsized":"2024-09-22T03:24:56.662965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1060,"log_dups_size":0,"ondisk_log_size":1060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1061,"num_write_kb":7680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"27'1072","reported_seq":1109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.885434+0000","last_change":"2024-09-22T03:23:16.067065+0000","last_active":"2024-09-22T03:24:56.885434+0000","last_peered":"2024-09-22T03:24:56.885434+0000","last_clean":"2024-09-22T03:24:56.885434+0000","last_became_active":"2024-09-22T03:23:16.057980+0000","last_became_peered":"2024-09-22T03:23:16.057980+0000","last_unstale":"2024-09-22T03:24:56.885434+0000","last_undegraded":"2024-09-22T03:24:56.885434+0000","last_fullsized":"2024-09-22T03:24:56.885434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1072,"log_dups_size":0,"ondisk_log_size":1072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1073,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"27'1108","reported_seq":1145,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.838808+0000","last_change":"2024-09-22T03:23:16.122149+0000","last_active":"2024-09-22T03:24:56.838808+0000","last_peered":"2024-09-22T03:24:56.838808+0000","last_clean":"2024-09-22T03:24:56.838808+0000","last_became_active":"2024-09-22T03:23:16.116606+0000","last_became_peered":"2024-09-22T03:23:16.116606+0000","last_unstale":"2024-09-22T03:24:56.838808+0000","last_undegraded":"2024-09-22T03:24:56.838808+0000","last_fullsized":"2024-09-22T03:24:56.838808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1109,"num_write_kb":8000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"27'1119","reported_seq":1156,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.882703+0000","last_change":"2024-09-22T03:23:16.064465+0000","last_active":"2024-09-22T03:24:56.882703+0000","last_peered":"2024-09-22T03:24:56.882703+0000","last_clean":"2024-09-22T03:24:56.882703+0000","last_became_active":"2024-09-22T03:23:16.060493+0000","last_became_peered":"2024-09-22T03:23:16.060493+0000","last_unstale":"2024-09-22T03:24:56.882703+0000","last_undegraded":"2024-09-22T03:24:56.882703+0000","last_fullsized":"2024-09-22T03:24:56.882703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"27'1121","reported_seq":1158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.865971+0000","last_change":"2024-09-22T03:23:16.134202+0000","last_active":"2024-09-22T03:24:56.865971+0000","last_peered":"2024-09-22T03:24:56.865971+0000","last_clean":"2024-09-22T03:24:56.865971+0000","last_became_active":"2024-09-22T03:23:16.130084+0000","last_became_peered":"2024-09-22T03:23:16.130084+0000","last_unstale":"2024-09-22T03:24:56.865971+0000","last_undegraded":"2024-09-22T03:24:56.865971+0000","last_fullsized":"2024-09-22T03:24:56.865971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1122,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.735395+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:24:34.735395+0000","last_peered":"2024-09-22T03:24:34.735395+0000","last_clean":"2024-09-22T03:24:34.735395+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:24:34.735395+0000","last_undegraded":"2024-09-22T03:24:34.735395+0000","last_fullsized":"2024-09-22T03:24:34.735395+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.729094+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:24:34.729094+0000","last_peered":"2024-09-22T03:24:34.729094+0000","last_clean":"2024-09-22T03:24:34.729094+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:24:34.729094+0000","last_undegraded":"2024-09-22T03:24:34.729094+0000","last_fullsized":"2024-09-22T03:24:34.729094+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"27'1149","reported_seq":1186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.890862+0000","last_change":"2024-09-22T03:23:16.054465+0000","last_active":"2024-09-22T03:24:56.890862+0000","last_peered":"2024-09-22T03:24:56.890862+0000","last_clean":"2024-09-22T03:24:56.890862+0000","last_became_active":"2024-09-22T03:23:16.045825+0000","last_became_peered":"2024-09-22T03:23:16.045825+0000","last_unstale":"2024-09-22T03:24:56.890862+0000","last_undegraded":"2024-09-22T03:24:56.890862+0000","last_fullsized":"2024-09-22T03:24:56.890862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1149,"log_dups_size":0,"ondisk_log_size":1149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.745191+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:24:34.745191+0000","last_peered":"2024-09-22T03:24:34.745191+0000","last_clean":"2024-09-22T03:24:34.745191+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:24:34.745191+0000","last_undegraded":"2024-09-22T03:24:34.745191+0000","last_fullsized":"2024-09-22T03:24:34.745191+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"27'990","reported_seq":1027,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.864786+0000","last_change":"2024-09-22T03:23:16.065728+0000","last_active":"2024-09-22T03:24:56.864786+0000","last_peered":"2024-09-22T03:24:56.864786+0000","last_clean":"2024-09-22T03:24:56.864786+0000","last_became_active":"2024-09-22T03:23:16.042326+0000","last_became_peered":"2024-09-22T03:23:16.042326+0000","last_unstale":"2024-09-22T03:24:56.864786+0000","last_undegraded":"2024-09-22T03:24:56.864786+0000","last_fullsized":"2024-09-22T03:24:56.864786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":991,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.724879+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:34.724879+0000","last_peered":"2024-09-22T03:24:34.724879+0000","last_clean":"2024-09-22T03:24:34.724879+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:24:34.724879+0000","last_undegraded":"2024-09-22T03:24:34.724879+0000","last_fullsized":"2024-09-22T03:24:34.724879+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'1221","reported_seq":1258,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.906696+0000","last_change":"2024-09-22T03:23:16.109755+0000","last_active":"2024-09-22T03:24:56.906696+0000","last_peered":"2024-09-22T03:24:56.906696+0000","last_clean":"2024-09-22T03:24:56.906696+0000","last_became_active":"2024-09-22T03:23:16.101645+0000","last_became_peered":"2024-09-22T03:23:16.101645+0000","last_unstale":"2024-09-22T03:24:56.906696+0000","last_undegraded":"2024-09-22T03:24:56.906696+0000","last_fullsized":"2024-09-22T03:24:56.906696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1221,"log_dups_size":0,"ondisk_log_size":1221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1222,"num_write_kb":8832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.740175+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:24:34.740175+0000","last_peered":"2024-09-22T03:24:34.740175+0000","last_clean":"2024-09-22T03:24:34.740175+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:24:34.740175+0000","last_undegraded":"2024-09-22T03:24:34.740175+0000","last_fullsized":"2024-09-22T03:24:34.740175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"27'997","reported_seq":1034,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.707651+0000","last_change":"2024-09-22T03:23:16.053472+0000","last_active":"2024-09-22T03:24:56.707651+0000","last_peered":"2024-09-22T03:24:56.707651+0000","last_clean":"2024-09-22T03:24:56.707651+0000","last_became_active":"2024-09-22T03:23:16.036115+0000","last_became_peered":"2024-09-22T03:23:16.036115+0000","last_unstale":"2024-09-22T03:24:56.707651+0000","last_undegraded":"2024-09-22T03:24:56.707651+0000","last_fullsized":"2024-09-22T03:24:56.707651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":998,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"27'1118","reported_seq":1155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.468674+0000","last_change":"2024-09-22T03:23:16.041533+0000","last_active":"2024-09-22T03:24:56.468674+0000","last_peered":"2024-09-22T03:24:56.468674+0000","last_clean":"2024-09-22T03:24:56.468674+0000","last_became_active":"2024-09-22T03:23:16.035027+0000","last_became_peered":"2024-09-22T03:23:16.035027+0000","last_unstale":"2024-09-22T03:24:56.468674+0000","last_undegraded":"2024-09-22T03:24:56.468674+0000","last_fullsized":"2024-09-22T03:24:56.468674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1119,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.720641+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:34.720641+0000","last_peered":"2024-09-22T03:24:34.720641+0000","last_clean":"2024-09-22T03:24:34.720641+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:24:34.720641+0000","last_undegraded":"2024-09-22T03:24:34.720641+0000","last_fullsized":"2024-09-22T03:24:34.720641+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"27'1286","reported_seq":1323,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.879959+0000","last_change":"2024-09-22T03:23:16.089014+0000","last_active":"2024-09-22T03:24:56.879959+0000","last_peered":"2024-09-22T03:24:56.879959+0000","last_clean":"2024-09-22T03:24:56.879959+0000","last_became_active":"2024-09-22T03:23:16.084308+0000","last_became_peered":"2024-09-22T03:23:16.084308+0000","last_unstale":"2024-09-22T03:24:56.879959+0000","last_undegraded":"2024-09-22T03:24:56.879959+0000","last_fullsized":"2024-09-22T03:24:56.879959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1286,"log_dups_size":0,"ondisk_log_size":1286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1287,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.745913+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:24:34.745913+0000","last_peered":"2024-09-22T03:24:34.745913+0000","last_clean":"2024-09-22T03:24:34.745913+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:24:34.745913+0000","last_undegraded":"2024-09-22T03:24:34.745913+0000","last_fullsized":"2024-09-22T03:24:34.745913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'1081","reported_seq":1118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.929197+0000","last_change":"2024-09-22T03:23:16.071985+0000","last_active":"2024-09-22T03:24:56.929197+0000","last_peered":"2024-09-22T03:24:56.929197+0000","last_clean":"2024-09-22T03:24:56.929197+0000","last_became_active":"2024-09-22T03:23:16.050412+0000","last_became_peered":"2024-09-22T03:23:16.050412+0000","last_unstale":"2024-09-22T03:24:56.929197+0000","last_undegraded":"2024-09-22T03:24:56.929197+0000","last_fullsized":"2024-09-22T03:24:56.929197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1082,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.720872+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:24:34.720872+0000","last_peered":"2024-09-22T03:24:34.720872+0000","last_clean":"2024-09-22T03:24:34.720872+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:24:34.720872+0000","last_undegraded":"2024-09-22T03:24:34.720872+0000","last_fullsized":"2024-09-22T03:24:34.720872+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'1276","reported_seq":1313,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:56.872824+0000","last_change":"2024-09-22T03:23:16.106036+0000","last_active":"2024-09-22T03:24:56.872824+0000","last_peered":"2024-09-22T03:24:56.872824+0000","last_clean":"2024-09-22T03:24:56.872824+0000","last_became_active":"2024-09-22T03:23:16.100761+0000","last_became_peered":"2024-09-22T03:23:16.100761+0000","last_unstale":"2024-09-22T03:24:56.872824+0000","last_undegraded":"2024-09-22T03:24:56.872824+0000","last_fullsized":"2024-09-22T03:24:56.872824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:14.959478+0000","last_clean_scrub_stamp":"2024-09-22T03:23:14.959478+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1277,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-22T03:24:34.728445+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:24:34.728445+0000","last_peered":"2024-09-22T03:24:34.728445+0000","last_clean":"2024-09-22T03:24:34.728445+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:24:34.728445+0000","last_undegraded":"2024-09-22T03:24:34.728445+0000","last_fullsized":"2024-09-22T03:24:34.728445+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":19398680,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17894,"num_write_kb":129345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214056751104,"internally_reserved":0,"allocated":154742784,"data_stored":154742784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17878,"ondisk_log_size":17878,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214056751104,"internally_reserved":0,"allocated":154742784,"data_stored":154742784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53506859008,"internally_reserved":0,"allocated":46014464,"data_stored":46014464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":47188,"kb_used_data":47188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52250540,"statfs":{"total":53552873472,"available":53504552960,"internally_reserved":0,"allocated":48320512,"data_stored":48320512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":37012,"kb_used_data":37012,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52260716,"statfs":{"total":53552873472,"available":53514973184,"internally_reserved":0,"allocated":37900288,"data_stored":37900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705779,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":39308,"kb_used_data":39308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52258420,"statfs":{"total":53552873472,"available":53512622080,"internally_reserved":0,"allocated":40251392,"data_stored":40251392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705779,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":35404,"kb_used_data":35404,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52262324,"statfs":{"total":53552873472,"available":53516619776,"internally_reserved":0,"allocated":36253696,"data_stored":36253696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53506859008,"internally_reserved":0,"allocated":46014464,"data_stored":46014464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53518725120,"internally_reserved":0,"allocated":34148352,"data_stored":34148352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53514485760,"internally_reserved":0,"allocated":38387712,"data_stored":38387712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53516681216,"internally_reserved":0,"allocated":36192256,"data_stored":36192256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53506859008,"internally_reserved":0,"allocated":46014464,"data_stored":46014464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53552873472,"available":53518725120,"internally_reserved":0,"allocated":34148352,"data_stored":34148352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":53552873472,"available":53514485760,"internally_reserved":0,"allocated":38387712,"data_stored":38387712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53552873472,"available":53516681216,"internally_reserved":0,"allocated":36192256,"data_stored":36192256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53552873472,"available":53506859008,"internally_reserved":0,"allocated":46014464,"data_stored":46014464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:24:58.682 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 2021 objects 2024-09-22T03:24:58.682 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :10.6076 2024-09-22T03:24:58.997 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:24:59.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:59.096+0000 7f0107458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:24:59.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:24:59.100+0000 7f0107458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:00.710 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-22T03:25:00.710 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-22T03:25:00.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:00.825+0000 7f9cf3597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:00.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:00.836+0000 7f9cf3597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:01.015 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_0' removed 2024-09-22T03:25:01.031 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:25:01.032 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:25:01.032 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-22T03:25:01.032 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_1 16 2024-09-22T03:25:01.169 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:01.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:01.169+0000 7fa803cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:01.184+0000 7fa803cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:01.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:01.284+0000 7fe2b6397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:01.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:01.289+0000 7fe2b6397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:02.026 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_1' created 2024-09-22T03:25:02.049 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-22T03:25:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:02.171+0000 7feca3997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:02.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:02.182+0000 7feca3997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.034 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-22T03:25:03.364 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:03.369 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:03.369 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:03.369 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:25:03.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:25:03.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.466+0000 7f6e1dd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.470+0000 7f6e1dd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.515+0000 7f4bab0fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.523+0000 7f4bab0fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:03.539 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:03.540 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:03.540 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:03.540 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:03.540 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:03.540 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:03.542 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:03.542 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:03.543 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:25:03.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.641+0000 7f514b6fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.649+0000 7f514b6fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:03.662 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:03.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:03.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:03.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:03.663 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:03.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:03.665 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:03.665 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:25:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.764+0000 7f3c4006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.772+0000 7f3c4006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:03.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:03.787 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:03.787 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:03.787 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:03.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:03.789 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:03.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:25:03.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.888+0000 7fc3eac4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:03.896+0000 7fc3eac4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:03.910 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:03.911 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:03.911 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:03.911 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:03.913 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:03.913 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:04.051 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-22T03:25:04.054 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:25:04.054 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:25:04.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:25:04.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:25:04.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_34331 2024-09-22T03:25:04.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:25:04.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:25:05.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 213 197 1.53898 1.53906 0.0343953 0.0775665 2024-09-22T03:25:05.550 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:05.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:05.686+0000 7f2e64d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:05.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:05.693+0000 7f2e64d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:06.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 401 385 1.5038 1.46875 0.0303714 0.0793411 2024-09-22T03:25:07.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 606 590 1.53634 1.60156 0.105671 0.0798958 2024-09-22T03:25:07.835 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:07.996+0000 7f0b37d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:08.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:08.003+0000 7f0b37d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:08.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 777 761 1.48621 1.33594 0.0374856 0.0821863 2024-09-22T03:25:08.914 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:08.914 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:08.914 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:25:08.914 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:25:09.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.059+0000 7f0e9ff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.071+0000 7f0e9ff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 1001 985 1.53893 1.75 0.0508183 0.0807444 2024-09-22T03:25:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:09.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:09.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:09.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:09.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:09.142 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:09.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:09.145 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:09.145 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:25:09.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.288+0000 7fe8ac22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.298+0000 7fe8ac22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.378 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:09.379 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:09.382 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:09.382 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:09.383 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:25:09.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.529+0000 7f8a76f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.543+0000 7f8a76f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.567 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:09.567 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:09.567 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:09.568 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:09.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:09.573 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:09.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:25:09.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.732+0000 7fa08b82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:09.744+0000 7fa08b82c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:09.799 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:09.823 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:09.823 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:09.823 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:09.823 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:09.824 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:10.110 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 1208 1192 1.55195 1.61719 0.150009 0.0801435 2024-09-22T03:25:10.219 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:10.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:10.337+0000 7f22caed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:10.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:10.343+0000 7f22caed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:11.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 1424 1408 1.57129 1.6875 0.101581 0.0790619 2024-09-22T03:25:12.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 16 1593 1577 1.53991 1.32031 0.102262 0.0804899 2024-09-22T03:25:12.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:12.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:12.726+0000 7fc2e0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:12.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:12.731+0000 7fc2e0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:13.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1763 1747 1.51636 1.32812 0.391744 0.0814294 2024-09-22T03:25:14.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 1960 1944 1.51862 1.53906 0.0753947 0.0817745 2024-09-22T03:25:14.824 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:14.824 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:14.824 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:25:14.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:25:14.879 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:14.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:14.964+0000 7f8b18517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:14.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:14.974+0000 7f8b18517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.019 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:15.019 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:15.020 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:15.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:15.024 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:15.025 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:25:15.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.031+0000 7f68e5c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.039+0000 7f68e5c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 2177 2161 1.53467 1.69531 0.121349 0.080975 2024-09-22T03:25:15.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.165+0000 7fa6c930e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.176+0000 7fa6c930e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:15.219 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:15.223 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:15.223 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:15.223 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:25:15.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.391+0000 7f581f797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.401+0000 7f581f797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.425 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:15.425 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:15.425 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:15.426 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:15.429 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:15.429 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:15.430 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:25:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.540+0000 7f33e3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:15.552+0000 7f33e3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:15.568 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:15.568 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:15.568 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:15.568 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:15.568 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:15.568 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:15.569 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:15.569 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:15.569 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:15.569 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:15.569 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:15.569 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:15.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:15.572 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:16.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 2359 2343 1.52526 1.42188 0.164884 0.0817272 2024-09-22T03:25:17.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 2578 2562 1.53953 1.71094 0.0766048 0.0809069 2024-09-22T03:25:17.240 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:17.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:17.401+0000 7ffbca66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:17.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:17.409+0000 7ffbca66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:18.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 2793 2777 1.54953 1.67969 0.0674444 0.0804435 2024-09-22T03:25:19.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 2979 2963 1.54309 1.45312 0.0393416 0.0808635 2024-09-22T03:25:19.628 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:19.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:19.762+0000 7fafb6d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:19.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:19.768+0000 7fafb6d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:20.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 3176 3160 1.54283 1.53906 0.129877 0.0808438 2024-09-22T03:25:20.573 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:20.573 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:20.573 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:25:20.574 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:25:20.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:20.719+0000 7f9171e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:20.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:20.729+0000 7f9171e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:20.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:20.787 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:20.787 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:20.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:20.789 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:20.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:25:20.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:20.903+0000 7f7335597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:20.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:20.913+0000 7f7335597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:20.975 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:20.975 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:20.975 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:20.976 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:20.979 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:20.979 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:20.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:25:21.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 3370 3354 1.54122 1.51562 0.0658011 0.0808061 2024-09-22T03:25:21.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:21.119+0000 7fbc79e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:21.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:21.129+0000 7fbc79e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:21.145 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:21.146 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:21.149 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:21.150 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:21.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:25:21.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:21.276+0000 7fdb04797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:21.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:21.285+0000 7fdb04797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:21.303 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:21.303 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:21.889 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:22.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:22.001+0000 7fc6e310e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:22.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:22.009+0000 7fc6e310e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:22.111 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 3544 3528 1.53112 1.35938 0.184055 0.0814358 2024-09-22T03:25:23.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 3744 3728 1.53276 1.5625 0.145306 0.0813316 2024-09-22T03:25:24.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:25:24.111440+0000 min lat: 0.0119124 max lat: 0.39705 avg lat: 0.0810893 2024-09-22T03:25:24.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:25:24.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 3948 3932 1.5358 1.59375 0.0919252 0.0810893 2024-09-22T03:25:24.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:24.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:24.251+0000 7f8594418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:24.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:24.256+0000 7f8594418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:25.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 4130 4114 1.53037 1.42188 0.0505909 0.0815271 2024-09-22T03:25:26.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 4315 4299 1.5265 1.44531 0.139039 0.0815859 2024-09-22T03:25:26.305 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:26.305 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:25:26.401 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:26.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:26.455+0000 7f1cb786c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:26.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:26.466+0000 7f1cb786c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:26.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:26.542+0000 7f85f4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:26.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:26.548+0000 7f85f4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:27.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 4489 4473 1.51923 1.35938 0.0788019 0.0821608 2024-09-22T03:25:27.731 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:25:27.751 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:25:27.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:27.865+0000 7f9080aba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:27.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:27.876+0000 7f9080aba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:28.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 4690 4674 1.52135 1.57031 0.151777 0.0820603 2024-09-22T03:25:28.742 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:25:28.761 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:25:28.787 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:28.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:28.895+0000 7f3daeb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:28.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:28.913+0000 7f3dad935640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:28.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:28.942+0000 7f0a32997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:28.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:28.948+0000 7f0a32997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:29.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 4890 4874 1.52299 1.5625 0.107601 0.0819576 2024-09-22T03:25:29.743 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:25:29.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:25:29.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:29.901+0000 7f62c5597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:29.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:29.912+0000 7f62c5597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:30.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 5079 5063 1.5212 1.47656 0.120994 0.0819488 2024-09-22T03:25:30.750 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:25:30.768 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:25:30.768 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:30.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:30.916+0000 7f593dd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:30.927+0000 7f593dd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 5284 5268 1.52417 1.60156 0.0843245 0.0818213 2024-09-22T03:25:31.116 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:31.225 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:25:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:25:31.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.234+0000 7f6f8ab57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.240+0000 7f6f8ab57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.249 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-22T03:25:30.749945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37044787,"num_objects":561,"num_object_clones":0,"num_object_copies":1683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4589,"num_write_kb":37496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4534,"ondisk_log_size":4534,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":225672,"kb_used_data":225672,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208965240,"statfs":{"total":214211493888,"available":213980405760,"internally_reserved":0,"allocated":231088128,"data_stored":231088128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7675904,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":7496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.787855"},"pg_stats":[{"pgid":"4.f","version":"31'271","reported_seq":284,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.743677+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:25:26.743677+0000","last_peered":"2024-09-22T03:25:26.743677+0000","last_clean":"2024-09-22T03:25:26.743677+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:25:26.743677+0000","last_undegraded":"2024-09-22T03:25:26.743677+0000","last_fullsized":"2024-09-22T03:25:26.743677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"31'223","reported_seq":236,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.892219+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:25:26.892219+0000","last_peered":"2024-09-22T03:25:26.892219+0000","last_clean":"2024-09-22T03:25:26.892219+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:25:26.892219+0000","last_undegraded":"2024-09-22T03:25:26.892219+0000","last_fullsized":"2024-09-22T03:25:26.892219+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"31'199","reported_seq":212,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.107145+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:25:26.107145+0000","last_peered":"2024-09-22T03:25:26.107145+0000","last_clean":"2024-09-22T03:25:26.107145+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:25:26.107145+0000","last_undegraded":"2024-09-22T03:25:26.107145+0000","last_fullsized":"2024-09-22T03:25:26.107145+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"31'239","reported_seq":252,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:24.893112+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:25:24.893112+0000","last_peered":"2024-09-22T03:25:24.893112+0000","last_clean":"2024-09-22T03:25:24.893112+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:25:24.893112+0000","last_undegraded":"2024-09-22T03:25:24.893112+0000","last_fullsized":"2024-09-22T03:25:24.893112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"31'335","reported_seq":348,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.368793+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:25:26.368793+0000","last_peered":"2024-09-22T03:25:26.368793+0000","last_clean":"2024-09-22T03:25:26.368793+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:25:26.368793+0000","last_undegraded":"2024-09-22T03:25:26.368793+0000","last_fullsized":"2024-09-22T03:25:26.368793+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"31'287","reported_seq":300,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.599671+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:25:26.599671+0000","last_peered":"2024-09-22T03:25:26.599671+0000","last_clean":"2024-09-22T03:25:26.599671+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:25:26.599671+0000","last_undegraded":"2024-09-22T03:25:26.599671+0000","last_fullsized":"2024-09-22T03:25:26.599671+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"31'287","reported_seq":300,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.260755+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:25:26.260755+0000","last_peered":"2024-09-22T03:25:26.260755+0000","last_clean":"2024-09-22T03:25:26.260755+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:25:26.260755+0000","last_undegraded":"2024-09-22T03:25:26.260755+0000","last_fullsized":"2024-09-22T03:25:26.260755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"31'343","reported_seq":356,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.843796+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:25:26.843796+0000","last_peered":"2024-09-22T03:25:26.843796+0000","last_clean":"2024-09-22T03:25:26.843796+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:25:26.843796+0000","last_undegraded":"2024-09-22T03:25:26.843796+0000","last_fullsized":"2024-09-22T03:25:26.843796+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'319","reported_seq":332,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.454732+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:25:26.454732+0000","last_peered":"2024-09-22T03:25:26.454732+0000","last_clean":"2024-09-22T03:25:26.454732+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:25:26.454732+0000","last_undegraded":"2024-09-22T03:25:26.454732+0000","last_fullsized":"2024-09-22T03:25:26.454732+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":196,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.052882+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:25:04.052882+0000","last_peered":"2024-09-22T03:25:04.052882+0000","last_clean":"2024-09-22T03:25:04.052882+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:25:04.052882+0000","last_undegraded":"2024-09-22T03:25:04.052882+0000","last_fullsized":"2024-09-22T03:25:04.052882+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.053927+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:25:04.053927+0000","last_peered":"2024-09-22T03:25:04.053927+0000","last_clean":"2024-09-22T03:25:04.053927+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:25:04.053927+0000","last_undegraded":"2024-09-22T03:25:04.053927+0000","last_fullsized":"2024-09-22T03:25:04.053927+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"31'298","reported_seq":311,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.946900+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:25:26.946900+0000","last_peered":"2024-09-22T03:25:26.946900+0000","last_clean":"2024-09-22T03:25:26.946900+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:25:26.946900+0000","last_undegraded":"2024-09-22T03:25:26.946900+0000","last_fullsized":"2024-09-22T03:25:26.946900+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2449408,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":2392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.055263+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:25:04.055263+0000","last_peered":"2024-09-22T03:25:04.055263+0000","last_clean":"2024-09-22T03:25:04.055263+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:25:04.055263+0000","last_undegraded":"2024-09-22T03:25:04.055263+0000","last_fullsized":"2024-09-22T03:25:04.055263+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"31'287","reported_seq":300,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.634607+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:25:26.634607+0000","last_peered":"2024-09-22T03:25:26.634607+0000","last_clean":"2024-09-22T03:25:26.634607+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:25:26.634607+0000","last_undegraded":"2024-09-22T03:25:26.634607+0000","last_fullsized":"2024-09-22T03:25:26.634607+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.050884+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:04.050884+0000","last_peered":"2024-09-22T03:25:04.050884+0000","last_clean":"2024-09-22T03:25:04.050884+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:25:04.050884+0000","last_undegraded":"2024-09-22T03:25:04.050884+0000","last_fullsized":"2024-09-22T03:25:04.050884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"31'231","reported_seq":244,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.660073+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:25:26.660073+0000","last_peered":"2024-09-22T03:25:26.660073+0000","last_clean":"2024-09-22T03:25:26.660073+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:25:26.660073+0000","last_undegraded":"2024-09-22T03:25:26.660073+0000","last_fullsized":"2024-09-22T03:25:26.660073+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.051084+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:04.051084+0000","last_peered":"2024-09-22T03:25:04.051084+0000","last_clean":"2024-09-22T03:25:04.051084+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:25:04.051084+0000","last_undegraded":"2024-09-22T03:25:04.051084+0000","last_fullsized":"2024-09-22T03:25:04.051084+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"31'231","reported_seq":244,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:25.165620+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:25:25.165620+0000","last_peered":"2024-09-22T03:25:25.165620+0000","last_clean":"2024-09-22T03:25:25.165620+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:25:25.165620+0000","last_undegraded":"2024-09-22T03:25:25.165620+0000","last_fullsized":"2024-09-22T03:25:25.165620+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.055454+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:04.055454+0000","last_peered":"2024-09-22T03:25:04.055454+0000","last_clean":"2024-09-22T03:25:04.055454+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:25:04.055454+0000","last_undegraded":"2024-09-22T03:25:04.055454+0000","last_fullsized":"2024-09-22T03:25:04.055454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"31'271","reported_seq":284,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:25.901639+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:25:25.901639+0000","last_peered":"2024-09-22T03:25:25.901639+0000","last_clean":"2024-09-22T03:25:25.901639+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:25:25.901639+0000","last_undegraded":"2024-09-22T03:25:25.901639+0000","last_fullsized":"2024-09-22T03:25:25.901639+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.056316+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:04.056316+0000","last_peered":"2024-09-22T03:25:04.056316+0000","last_clean":"2024-09-22T03:25:04.056316+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:25:04.056316+0000","last_undegraded":"2024-09-22T03:25:04.056316+0000","last_fullsized":"2024-09-22T03:25:04.056316+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'358","reported_seq":371,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.945083+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:25:26.945083+0000","last_peered":"2024-09-22T03:25:26.945083+0000","last_clean":"2024-09-22T03:25:26.945083+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:25:26.945083+0000","last_undegraded":"2024-09-22T03:25:26.945083+0000","last_fullsized":"2024-09-22T03:25:26.945083+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":359,"num_write_kb":2872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.053807+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:25:04.053807+0000","last_peered":"2024-09-22T03:25:04.053807+0000","last_clean":"2024-09-22T03:25:04.053807+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:25:04.053807+0000","last_undegraded":"2024-09-22T03:25:04.053807+0000","last_fullsized":"2024-09-22T03:25:04.053807+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"31'255","reported_seq":268,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.488003+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:25:26.488003+0000","last_peered":"2024-09-22T03:25:26.488003+0000","last_clean":"2024-09-22T03:25:26.488003+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:25:26.488003+0000","last_undegraded":"2024-09-22T03:25:26.488003+0000","last_fullsized":"2024-09-22T03:25:26.488003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.053423+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:25:04.053423+0000","last_peered":"2024-09-22T03:25:04.053423+0000","last_clean":"2024-09-22T03:25:04.053423+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:25:04.053423+0000","last_undegraded":"2024-09-22T03:25:04.053423+0000","last_fullsized":"2024-09-22T03:25:04.053423+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":36454400,"num_objects":557,"num_object_clones":0,"num_object_copies":1671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":35600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213979471872,"internally_reserved":0,"allocated":232022016,"data_stored":232022016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4434,"ondisk_log_size":4434,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213979471872,"internally_reserved":0,"allocated":232022016,"data_stored":232022016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":61384,"kb_used_data":61384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52236344,"statfs":{"total":53552873472,"available":53490016256,"internally_reserved":0,"allocated":62857216,"data_stored":62857216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":45304,"kb_used_data":45304,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52252424,"statfs":{"total":53552873472,"available":53506482176,"internally_reserved":0,"allocated":46391296,"data_stored":46391296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":59900,"kb_used_data":59900,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52237828,"statfs":{"total":53552873472,"available":53491535872,"internally_reserved":0,"allocated":61337600,"data_stored":61337600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":59084,"kb_used_data":59084,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52238644,"statfs":{"total":53552873472,"available":53492371456,"internally_reserved":0,"allocated":60502016,"data_stored":60502016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53492174848,"internally_reserved":0,"allocated":60698624,"data_stored":60698624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53491277824,"internally_reserved":0,"allocated":61595648,"data_stored":61595648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53506289664,"internally_reserved":0,"allocated":46583808,"data_stored":46583808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53492174848,"internally_reserved":0,"allocated":60698624,"data_stored":60698624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53491277824,"internally_reserved":0,"allocated":61595648,"data_stored":61595648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53506289664,"internally_reserved":0,"allocated":46583808,"data_stored":46583808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:31.250 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:31.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.403+0000 7f767df17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.414+0000 7f767df17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.626 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:25:31.626 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:25:31.641 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-22T03:25:30.749945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37044787,"num_objects":561,"num_object_clones":0,"num_object_copies":1683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4589,"num_write_kb":37496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4534,"ondisk_log_size":4534,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":225672,"kb_used_data":225672,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208965240,"statfs":{"total":214211493888,"available":213980405760,"internally_reserved":0,"allocated":231088128,"data_stored":231088128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7675904,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":7496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.787855"},"pg_stats":[{"pgid":"4.f","version":"31'271","reported_seq":284,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.743677+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:25:26.743677+0000","last_peered":"2024-09-22T03:25:26.743677+0000","last_clean":"2024-09-22T03:25:26.743677+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:25:26.743677+0000","last_undegraded":"2024-09-22T03:25:26.743677+0000","last_fullsized":"2024-09-22T03:25:26.743677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"31'223","reported_seq":236,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.892219+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:25:26.892219+0000","last_peered":"2024-09-22T03:25:26.892219+0000","last_clean":"2024-09-22T03:25:26.892219+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:25:26.892219+0000","last_undegraded":"2024-09-22T03:25:26.892219+0000","last_fullsized":"2024-09-22T03:25:26.892219+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"31'199","reported_seq":212,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.107145+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:25:26.107145+0000","last_peered":"2024-09-22T03:25:26.107145+0000","last_clean":"2024-09-22T03:25:26.107145+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:25:26.107145+0000","last_undegraded":"2024-09-22T03:25:26.107145+0000","last_fullsized":"2024-09-22T03:25:26.107145+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"31'239","reported_seq":252,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:24.893112+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:25:24.893112+0000","last_peered":"2024-09-22T03:25:24.893112+0000","last_clean":"2024-09-22T03:25:24.893112+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:25:24.893112+0000","last_undegraded":"2024-09-22T03:25:24.893112+0000","last_fullsized":"2024-09-22T03:25:24.893112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"31'335","reported_seq":348,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.368793+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:25:26.368793+0000","last_peered":"2024-09-22T03:25:26.368793+0000","last_clean":"2024-09-22T03:25:26.368793+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:25:26.368793+0000","last_undegraded":"2024-09-22T03:25:26.368793+0000","last_fullsized":"2024-09-22T03:25:26.368793+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"31'287","reported_seq":300,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.599671+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:25:26.599671+0000","last_peered":"2024-09-22T03:25:26.599671+0000","last_clean":"2024-09-22T03:25:26.599671+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:25:26.599671+0000","last_undegraded":"2024-09-22T03:25:26.599671+0000","last_fullsized":"2024-09-22T03:25:26.599671+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"31'287","reported_seq":300,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.260755+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:25:26.260755+0000","last_peered":"2024-09-22T03:25:26.260755+0000","last_clean":"2024-09-22T03:25:26.260755+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:25:26.260755+0000","last_undegraded":"2024-09-22T03:25:26.260755+0000","last_fullsized":"2024-09-22T03:25:26.260755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"31'343","reported_seq":356,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.843796+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:25:26.843796+0000","last_peered":"2024-09-22T03:25:26.843796+0000","last_clean":"2024-09-22T03:25:26.843796+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:25:26.843796+0000","last_undegraded":"2024-09-22T03:25:26.843796+0000","last_fullsized":"2024-09-22T03:25:26.843796+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'319","reported_seq":332,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.454732+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:25:26.454732+0000","last_peered":"2024-09-22T03:25:26.454732+0000","last_clean":"2024-09-22T03:25:26.454732+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:25:26.454732+0000","last_undegraded":"2024-09-22T03:25:26.454732+0000","last_fullsized":"2024-09-22T03:25:26.454732+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":196,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.052882+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:25:04.052882+0000","last_peered":"2024-09-22T03:25:04.052882+0000","last_clean":"2024-09-22T03:25:04.052882+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:25:04.052882+0000","last_undegraded":"2024-09-22T03:25:04.052882+0000","last_fullsized":"2024-09-22T03:25:04.052882+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.053927+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:25:04.053927+0000","last_peered":"2024-09-22T03:25:04.053927+0000","last_clean":"2024-09-22T03:25:04.053927+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:25:04.053927+0000","last_undegraded":"2024-09-22T03:25:04.053927+0000","last_fullsized":"2024-09-22T03:25:04.053927+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"31'298","reported_seq":311,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.946900+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:25:26.946900+0000","last_peered":"2024-09-22T03:25:26.946900+0000","last_clean":"2024-09-22T03:25:26.946900+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:25:26.946900+0000","last_undegraded":"2024-09-22T03:25:26.946900+0000","last_fullsized":"2024-09-22T03:25:26.946900+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2449408,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":2392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.055263+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:25:04.055263+0000","last_peered":"2024-09-22T03:25:04.055263+0000","last_clean":"2024-09-22T03:25:04.055263+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:25:04.055263+0000","last_undegraded":"2024-09-22T03:25:04.055263+0000","last_fullsized":"2024-09-22T03:25:04.055263+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"31'287","reported_seq":300,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.634607+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:25:26.634607+0000","last_peered":"2024-09-22T03:25:26.634607+0000","last_clean":"2024-09-22T03:25:26.634607+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:25:26.634607+0000","last_undegraded":"2024-09-22T03:25:26.634607+0000","last_fullsized":"2024-09-22T03:25:26.634607+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.050884+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:04.050884+0000","last_peered":"2024-09-22T03:25:04.050884+0000","last_clean":"2024-09-22T03:25:04.050884+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:25:04.050884+0000","last_undegraded":"2024-09-22T03:25:04.050884+0000","last_fullsized":"2024-09-22T03:25:04.050884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"31'231","reported_seq":244,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.660073+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:25:26.660073+0000","last_peered":"2024-09-22T03:25:26.660073+0000","last_clean":"2024-09-22T03:25:26.660073+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:25:26.660073+0000","last_undegraded":"2024-09-22T03:25:26.660073+0000","last_fullsized":"2024-09-22T03:25:26.660073+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.051084+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:04.051084+0000","last_peered":"2024-09-22T03:25:04.051084+0000","last_clean":"2024-09-22T03:25:04.051084+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:25:04.051084+0000","last_undegraded":"2024-09-22T03:25:04.051084+0000","last_fullsized":"2024-09-22T03:25:04.051084+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"31'231","reported_seq":244,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:25.165620+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:25:25.165620+0000","last_peered":"2024-09-22T03:25:25.165620+0000","last_clean":"2024-09-22T03:25:25.165620+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:25:25.165620+0000","last_undegraded":"2024-09-22T03:25:25.165620+0000","last_fullsized":"2024-09-22T03:25:25.165620+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.055454+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:04.055454+0000","last_peered":"2024-09-22T03:25:04.055454+0000","last_clean":"2024-09-22T03:25:04.055454+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:25:04.055454+0000","last_undegraded":"2024-09-22T03:25:04.055454+0000","last_fullsized":"2024-09-22T03:25:04.055454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"31'271","reported_seq":284,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:25.901639+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:25:25.901639+0000","last_peered":"2024-09-22T03:25:25.901639+0000","last_clean":"2024-09-22T03:25:25.901639+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:25:25.901639+0000","last_undegraded":"2024-09-22T03:25:25.901639+0000","last_fullsized":"2024-09-22T03:25:25.901639+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.056316+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:04.056316+0000","last_peered":"2024-09-22T03:25:04.056316+0000","last_clean":"2024-09-22T03:25:04.056316+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:25:04.056316+0000","last_undegraded":"2024-09-22T03:25:04.056316+0000","last_fullsized":"2024-09-22T03:25:04.056316+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'358","reported_seq":371,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.945083+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:25:26.945083+0000","last_peered":"2024-09-22T03:25:26.945083+0000","last_clean":"2024-09-22T03:25:26.945083+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:25:26.945083+0000","last_undegraded":"2024-09-22T03:25:26.945083+0000","last_fullsized":"2024-09-22T03:25:26.945083+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":359,"num_write_kb":2872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.053807+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:25:04.053807+0000","last_peered":"2024-09-22T03:25:04.053807+0000","last_clean":"2024-09-22T03:25:04.053807+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:25:04.053807+0000","last_undegraded":"2024-09-22T03:25:04.053807+0000","last_fullsized":"2024-09-22T03:25:04.053807+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"31'255","reported_seq":268,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:26.488003+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:25:26.488003+0000","last_peered":"2024-09-22T03:25:26.488003+0000","last_clean":"2024-09-22T03:25:26.488003+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:25:26.488003+0000","last_undegraded":"2024-09-22T03:25:26.488003+0000","last_fullsized":"2024-09-22T03:25:26.488003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T03:25:04.053423+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:25:04.053423+0000","last_peered":"2024-09-22T03:25:04.053423+0000","last_clean":"2024-09-22T03:25:04.053423+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:25:04.053423+0000","last_undegraded":"2024-09-22T03:25:04.053423+0000","last_fullsized":"2024-09-22T03:25:04.053423+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":36454400,"num_objects":557,"num_object_clones":0,"num_object_copies":1671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":35600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213979471872,"internally_reserved":0,"allocated":232022016,"data_stored":232022016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4434,"ondisk_log_size":4434,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213979471872,"internally_reserved":0,"allocated":232022016,"data_stored":232022016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":61384,"kb_used_data":61384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52236344,"statfs":{"total":53552873472,"available":53490016256,"internally_reserved":0,"allocated":62857216,"data_stored":62857216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":45304,"kb_used_data":45304,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52252424,"statfs":{"total":53552873472,"available":53506482176,"internally_reserved":0,"allocated":46391296,"data_stored":46391296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":59900,"kb_used_data":59900,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52237828,"statfs":{"total":53552873472,"available":53491535872,"internally_reserved":0,"allocated":61337600,"data_stored":61337600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":59084,"kb_used_data":59084,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52238644,"statfs":{"total":53552873472,"available":53492371456,"internally_reserved":0,"allocated":60502016,"data_stored":60502016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53492174848,"internally_reserved":0,"allocated":60698624,"data_stored":60698624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53491277824,"internally_reserved":0,"allocated":61595648,"data_stored":61595648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53506289664,"internally_reserved":0,"allocated":46583808,"data_stored":46583808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53492174848,"internally_reserved":0,"allocated":60698624,"data_stored":60698624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53491277824,"internally_reserved":0,"allocated":61595648,"data_stored":61595648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53506289664,"internally_reserved":0,"allocated":46583808,"data_stored":46583808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53489729536,"internally_reserved":0,"allocated":63143936,"data_stored":63143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:31.642 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:25:31.642 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:31.643 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:25:31.643 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:25:31.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.770+0000 7fce67757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.780+0000 7fce67757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:31.799 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:31.799 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:31.799 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:31.799 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:31.801 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:31.801 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:31.802 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:25:31.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.933+0000 7f80648b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:31.948+0000 7f80648b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:31.978 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:31.979 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:31.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:31.982 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:31.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:25:32.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:32.094+0000 7f5883e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:32.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:32.103+0000 7f5883e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:32.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 5444 5428 1.51438 1.25 0.102667 0.082371 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:32.124 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:32.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:32.128 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:32.128 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:25:32.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:32.288+0000 7fb17e797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:32.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:32.298+0000 7fb17e797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:32.347 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:32.348 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:32.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:32.352 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:33.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 5628 5612 1.51172 1.4375 0.0531156 0.0825754 2024-09-22T03:25:33.469 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:33.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:33.618+0000 7f3db5317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:33.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:33.624+0000 7f3db5317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:34.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 5799 5783 1.50586 1.33594 0.162802 0.0828951 2024-09-22T03:25:35.112 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 6006 5990 1.50944 1.61719 0.0613823 0.082678 2024-09-22T03:25:35.801 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:35.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:35.938+0000 7f316b8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:35.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:35.945+0000 7f316b8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:36.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 16 6184 6168 1.50573 1.39062 0.0584146 0.0828854 2024-09-22T03:25:37.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 6384 6368 1.50744 1.5625 0.0468279 0.0828001 2024-09-22T03:25:37.353 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:37.353 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:37.354 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:37.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:37.467+0000 7f826f317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:37.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:37.476+0000 7f826f317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:37.741 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:25:37.742 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:25:37.758 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-22T03:25:35.963575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":44900915,"num_objects":681,"num_object_clones":0,"num_object_copies":2043,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5548,"num_write_kb":45168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5495,"ondisk_log_size":5495,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":274152,"kb_used_data":274152,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208916760,"statfs":{"total":214211493888,"available":213930762240,"internally_reserved":0,"allocated":280731648,"data_stored":280731648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7856128,"num_objects":120,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":959,"num_write_kb":7672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.221209"},"pg_stats":[{"pgid":"4.f","version":"35'351","reported_seq":376,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.772957+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:25:31.772957+0000","last_peered":"2024-09-22T03:25:31.772957+0000","last_clean":"2024-09-22T03:25:31.772957+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:25:31.772957+0000","last_undegraded":"2024-09-22T03:25:31.772957+0000","last_fullsized":"2024-09-22T03:25:31.772957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"35'303","reported_seq":328,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.124251+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:25:31.124251+0000","last_peered":"2024-09-22T03:25:31.124251+0000","last_clean":"2024-09-22T03:25:31.124251+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:25:31.124251+0000","last_undegraded":"2024-09-22T03:25:31.124251+0000","last_fullsized":"2024-09-22T03:25:31.124251+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"35'263","reported_seq":288,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.929164+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:25:31.929164+0000","last_peered":"2024-09-22T03:25:31.929164+0000","last_clean":"2024-09-22T03:25:31.929164+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:25:31.929164+0000","last_undegraded":"2024-09-22T03:25:31.929164+0000","last_fullsized":"2024-09-22T03:25:31.929164+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"35'335","reported_seq":360,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.190633+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:25:31.190633+0000","last_peered":"2024-09-22T03:25:31.190633+0000","last_clean":"2024-09-22T03:25:31.190633+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:25:31.190633+0000","last_undegraded":"2024-09-22T03:25:31.190633+0000","last_fullsized":"2024-09-22T03:25:31.190633+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"35'367","reported_seq":392,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.064872+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:25:31.064872+0000","last_peered":"2024-09-22T03:25:31.064872+0000","last_clean":"2024-09-22T03:25:31.064872+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:25:31.064872+0000","last_undegraded":"2024-09-22T03:25:31.064872+0000","last_fullsized":"2024-09-22T03:25:31.064872+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"34'360","reported_seq":384,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.803294+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:25:30.803294+0000","last_peered":"2024-09-22T03:25:30.803294+0000","last_clean":"2024-09-22T03:25:30.803294+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:25:30.803294+0000","last_undegraded":"2024-09-22T03:25:30.803294+0000","last_fullsized":"2024-09-22T03:25:30.803294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"35'351","reported_seq":376,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.744810+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:25:31.744810+0000","last_peered":"2024-09-22T03:25:31.744810+0000","last_clean":"2024-09-22T03:25:31.744810+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:25:31.744810+0000","last_undegraded":"2024-09-22T03:25:31.744810+0000","last_fullsized":"2024-09-22T03:25:31.744810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"35'407","reported_seq":432,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.963569+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:25:30.963569+0000","last_peered":"2024-09-22T03:25:30.963569+0000","last_clean":"2024-09-22T03:25:30.963569+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:25:30.963569+0000","last_undegraded":"2024-09-22T03:25:30.963569+0000","last_fullsized":"2024-09-22T03:25:30.963569+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"34'368","reported_seq":392,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.799995+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:25:30.799995+0000","last_peered":"2024-09-22T03:25:30.799995+0000","last_clean":"2024-09-22T03:25:30.799995+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:25:30.799995+0000","last_undegraded":"2024-09-22T03:25:30.799995+0000","last_fullsized":"2024-09-22T03:25:30.799995+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":208,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.771791+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:25:30.771791+0000","last_peered":"2024-09-22T03:25:30.771791+0000","last_clean":"2024-09-22T03:25:30.771791+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:25:30.771791+0000","last_undegraded":"2024-09-22T03:25:30.771791+0000","last_fullsized":"2024-09-22T03:25:30.771791+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":74,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.802001+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:25:30.802001+0000","last_peered":"2024-09-22T03:25:30.802001+0000","last_clean":"2024-09-22T03:25:30.802001+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:25:30.802001+0000","last_undegraded":"2024-09-22T03:25:30.802001+0000","last_fullsized":"2024-09-22T03:25:30.802001+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"35'351","reported_seq":376,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.624231+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:25:31.624231+0000","last_peered":"2024-09-22T03:25:31.624231+0000","last_clean":"2024-09-22T03:25:31.624231+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:25:31.624231+0000","last_undegraded":"2024-09-22T03:25:31.624231+0000","last_fullsized":"2024-09-22T03:25:31.624231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.869396+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:25:30.869396+0000","last_peered":"2024-09-22T03:25:30.869396+0000","last_clean":"2024-09-22T03:25:30.869396+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:25:30.869396+0000","last_undegraded":"2024-09-22T03:25:30.869396+0000","last_fullsized":"2024-09-22T03:25:30.869396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"35'343","reported_seq":368,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.296930+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:25:31.296930+0000","last_peered":"2024-09-22T03:25:31.296930+0000","last_clean":"2024-09-22T03:25:31.296930+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:25:31.296930+0000","last_undegraded":"2024-09-22T03:25:31.296930+0000","last_fullsized":"2024-09-22T03:25:31.296930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.798581+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:30.798581+0000","last_peered":"2024-09-22T03:25:30.798581+0000","last_clean":"2024-09-22T03:25:30.798581+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:25:30.798581+0000","last_undegraded":"2024-09-22T03:25:30.798581+0000","last_fullsized":"2024-09-22T03:25:30.798581+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"35'304","reported_seq":329,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.831596+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:25:31.831596+0000","last_peered":"2024-09-22T03:25:31.831596+0000","last_clean":"2024-09-22T03:25:31.831596+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:25:31.831596+0000","last_undegraded":"2024-09-22T03:25:31.831596+0000","last_fullsized":"2024-09-22T03:25:31.831596+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2498560,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":2440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":75,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.864449+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:30.864449+0000","last_peered":"2024-09-22T03:25:30.864449+0000","last_clean":"2024-09-22T03:25:30.864449+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:25:30.864449+0000","last_undegraded":"2024-09-22T03:25:30.864449+0000","last_fullsized":"2024-09-22T03:25:30.864449+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"35'287","reported_seq":312,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.424940+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:25:31.424940+0000","last_peered":"2024-09-22T03:25:31.424940+0000","last_clean":"2024-09-22T03:25:31.424940+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:25:31.424940+0000","last_undegraded":"2024-09-22T03:25:31.424940+0000","last_fullsized":"2024-09-22T03:25:31.424940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.772381+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:30.772381+0000","last_peered":"2024-09-22T03:25:30.772381+0000","last_clean":"2024-09-22T03:25:30.772381+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:25:30.772381+0000","last_undegraded":"2024-09-22T03:25:30.772381+0000","last_fullsized":"2024-09-22T03:25:30.772381+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"35'335","reported_seq":360,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.304993+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:25:31.304993+0000","last_peered":"2024-09-22T03:25:31.304993+0000","last_clean":"2024-09-22T03:25:31.304993+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:25:31.304993+0000","last_undegraded":"2024-09-22T03:25:31.304993+0000","last_fullsized":"2024-09-22T03:25:31.304993+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.774266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:30.774266+0000","last_peered":"2024-09-22T03:25:30.774266+0000","last_clean":"2024-09-22T03:25:30.774266+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:25:30.774266+0000","last_undegraded":"2024-09-22T03:25:30.774266+0000","last_fullsized":"2024-09-22T03:25:30.774266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"35'391","reported_seq":416,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.919549+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:25:31.919549+0000","last_peered":"2024-09-22T03:25:31.919549+0000","last_clean":"2024-09-22T03:25:31.919549+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:25:31.919549+0000","last_undegraded":"2024-09-22T03:25:31.919549+0000","last_fullsized":"2024-09-22T03:25:31.919549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.771115+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:25:30.771115+0000","last_peered":"2024-09-22T03:25:30.771115+0000","last_clean":"2024-09-22T03:25:30.771115+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:25:30.771115+0000","last_undegraded":"2024-09-22T03:25:30.771115+0000","last_fullsized":"2024-09-22T03:25:30.771115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"35'279","reported_seq":304,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:31.531416+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:25:31.531416+0000","last_peered":"2024-09-22T03:25:31.531416+0000","last_clean":"2024-09-22T03:25:31.531416+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:25:31.531416+0000","last_undegraded":"2024-09-22T03:25:31.531416+0000","last_fullsized":"2024-09-22T03:25:31.531416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.802359+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:25:30.802359+0000","last_peered":"2024-09-22T03:25:30.802359+0000","last_clean":"2024-09-22T03:25:30.802359+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:25:30.802359+0000","last_undegraded":"2024-09-22T03:25:30.802359+0000","last_fullsized":"2024-09-22T03:25:30.802359+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":44310528,"num_objects":677,"num_object_clones":0,"num_object_copies":2031,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5409,"num_write_kb":43272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213930000384,"internally_reserved":0,"allocated":281493504,"data_stored":281493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5395,"ondisk_log_size":5395,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213930008576,"internally_reserved":0,"allocated":281485312,"data_stored":281485312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53475999744,"internally_reserved":0,"allocated":76873728,"data_stored":76873728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":74856,"kb_used_data":74856,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52222872,"statfs":{"total":53552873472,"available":53476220928,"internally_reserved":0,"allocated":76652544,"data_stored":76652544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":55876,"kb_used_data":55876,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52241852,"statfs":{"total":53552873472,"available":53495656448,"internally_reserved":0,"allocated":57217024,"data_stored":57217024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":73040,"kb_used_data":73040,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52224688,"statfs":{"total":53552873472,"available":53478080512,"internally_reserved":0,"allocated":74792960,"data_stored":74792960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":70380,"kb_used_data":70380,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52227348,"statfs":{"total":53552873472,"available":53480804352,"internally_reserved":0,"allocated":72069120,"data_stored":72069120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53475999744,"internally_reserved":0,"allocated":76873728,"data_stored":76873728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53480656896,"internally_reserved":0,"allocated":72216576,"data_stored":72216576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53477842944,"internally_reserved":0,"allocated":75030528,"data_stored":75030528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53495508992,"internally_reserved":0,"allocated":57364480,"data_stored":57364480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53475999744,"internally_reserved":0,"allocated":76873728,"data_stored":76873728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53480656896,"internally_reserved":0,"allocated":72216576,"data_stored":72216576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53477842944,"internally_reserved":0,"allocated":75030528,"data_stored":75030528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53495500800,"internally_reserved":0,"allocated":57372672,"data_stored":57372672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53475999744,"internally_reserved":0,"allocated":76873728,"data_stored":76873728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:38.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 6575 6559 1.50699 1.49219 0.0563367 0.0828569 2024-09-22T03:25:38.172 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:38.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:38.304+0000 7f335066c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:38.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:38.310+0000 7f335066c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:39.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 6771 6755 1.50768 1.53125 0.122332 0.0828088 2024-09-22T03:25:40.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 6950 6934 1.50464 1.39844 0.0499408 0.0829547 2024-09-22T03:25:40.426 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:40.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:40.579+0000 7f1fb1f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:40.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:40.587+0000 7f1fb1f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:41.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 7145 7129 1.50515 1.52344 0.10747 0.0829701 2024-09-22T03:25:42.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 7352 7336 1.50809 1.61719 0.110066 0.0827577 2024-09-22T03:25:42.760 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:42.760 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:42.760 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:42.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:42.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:42.889+0000 7efd1b4fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:42.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:42.899+0000 7efd1b4fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:43.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:43.037+0000 7fe1d3e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:43.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:43.046+0000 7fe1d3e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:43.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 7540 7524 1.50708 1.46875 0.0689153 0.0828112 2024-09-22T03:25:43.241 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:25:43.241 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:25:43.255 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-22T03:25:41.964446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":56861235,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":7008,"num_write_kb":56848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6953,"ondisk_log_size":6953,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":356536,"kb_used_data":356536,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208834376,"statfs":{"total":214211493888,"available":213846401024,"internally_reserved":0,"allocated":365092864,"data_stored":365092864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19816448,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2419,"num_write_kb":19352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.214500"},"pg_stats":[{"pgid":"4.f","version":"35'479","reported_seq":504,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:41.738392+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:25:41.738392+0000","last_peered":"2024-09-22T03:25:41.738392+0000","last_clean":"2024-09-22T03:25:41.738392+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:25:41.738392+0000","last_undegraded":"2024-09-22T03:25:41.738392+0000","last_fullsized":"2024-09-22T03:25:41.738392+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"35'367","reported_seq":392,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:41.833036+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:25:41.833036+0000","last_peered":"2024-09-22T03:25:41.833036+0000","last_clean":"2024-09-22T03:25:41.833036+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:25:41.833036+0000","last_undegraded":"2024-09-22T03:25:41.833036+0000","last_fullsized":"2024-09-22T03:25:41.833036+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"35'335","reported_seq":360,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:36.165164+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:25:36.165164+0000","last_peered":"2024-09-22T03:25:36.165164+0000","last_clean":"2024-09-22T03:25:36.165164+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:25:36.165164+0000","last_undegraded":"2024-09-22T03:25:36.165164+0000","last_fullsized":"2024-09-22T03:25:36.165164+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"35'471","reported_seq":496,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:41.144269+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:25:41.144269+0000","last_peered":"2024-09-22T03:25:41.144269+0000","last_clean":"2024-09-22T03:25:41.144269+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:25:41.144269+0000","last_undegraded":"2024-09-22T03:25:41.144269+0000","last_fullsized":"2024-09-22T03:25:41.144269+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"35'471","reported_seq":496,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:37.345462+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:25:37.345462+0000","last_peered":"2024-09-22T03:25:37.345462+0000","last_clean":"2024-09-22T03:25:37.345462+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:25:37.345462+0000","last_undegraded":"2024-09-22T03:25:37.345462+0000","last_fullsized":"2024-09-22T03:25:37.345462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"35'399","reported_seq":424,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:36.737658+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:25:36.737658+0000","last_peered":"2024-09-22T03:25:36.737658+0000","last_clean":"2024-09-22T03:25:36.737658+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:25:36.737658+0000","last_undegraded":"2024-09-22T03:25:36.737658+0000","last_fullsized":"2024-09-22T03:25:36.737658+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"35'391","reported_seq":416,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:36.534963+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:25:36.534963+0000","last_peered":"2024-09-22T03:25:36.534963+0000","last_clean":"2024-09-22T03:25:36.534963+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:25:36.534963+0000","last_undegraded":"2024-09-22T03:25:36.534963+0000","last_fullsized":"2024-09-22T03:25:36.534963+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"35'487","reported_seq":512,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:36.698227+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:25:36.698227+0000","last_peered":"2024-09-22T03:25:36.698227+0000","last_clean":"2024-09-22T03:25:36.698227+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:25:36.698227+0000","last_undegraded":"2024-09-22T03:25:36.698227+0000","last_fullsized":"2024-09-22T03:25:36.698227+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"35'423","reported_seq":448,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:36.934162+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:25:36.934162+0000","last_peered":"2024-09-22T03:25:36.934162+0000","last_clean":"2024-09-22T03:25:36.934162+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:25:36.934162+0000","last_undegraded":"2024-09-22T03:25:36.934162+0000","last_fullsized":"2024-09-22T03:25:36.934162+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":208,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.771791+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:25:30.771791+0000","last_peered":"2024-09-22T03:25:30.771791+0000","last_clean":"2024-09-22T03:25:30.771791+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:25:30.771791+0000","last_undegraded":"2024-09-22T03:25:30.771791+0000","last_fullsized":"2024-09-22T03:25:30.771791+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":74,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.802001+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:25:30.802001+0000","last_peered":"2024-09-22T03:25:30.802001+0000","last_clean":"2024-09-22T03:25:30.802001+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:25:30.802001+0000","last_undegraded":"2024-09-22T03:25:30.802001+0000","last_fullsized":"2024-09-22T03:25:30.802001+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"35'463","reported_seq":488,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:41.807044+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:25:41.807044+0000","last_peered":"2024-09-22T03:25:41.807044+0000","last_clean":"2024-09-22T03:25:41.807044+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:25:41.807044+0000","last_undegraded":"2024-09-22T03:25:41.807044+0000","last_fullsized":"2024-09-22T03:25:41.807044+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.869396+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:25:30.869396+0000","last_peered":"2024-09-22T03:25:30.869396+0000","last_clean":"2024-09-22T03:25:30.869396+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:25:30.869396+0000","last_undegraded":"2024-09-22T03:25:30.869396+0000","last_fullsized":"2024-09-22T03:25:30.869396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"35'463","reported_seq":488,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:41.888615+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:25:41.888615+0000","last_peered":"2024-09-22T03:25:41.888615+0000","last_clean":"2024-09-22T03:25:41.888615+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:25:41.888615+0000","last_undegraded":"2024-09-22T03:25:41.888615+0000","last_fullsized":"2024-09-22T03:25:41.888615+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.798581+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:30.798581+0000","last_peered":"2024-09-22T03:25:30.798581+0000","last_clean":"2024-09-22T03:25:30.798581+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:25:30.798581+0000","last_undegraded":"2024-09-22T03:25:30.798581+0000","last_fullsized":"2024-09-22T03:25:30.798581+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"35'407","reported_seq":432,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:39.298663+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:25:39.298663+0000","last_peered":"2024-09-22T03:25:39.298663+0000","last_clean":"2024-09-22T03:25:39.298663+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:25:39.298663+0000","last_undegraded":"2024-09-22T03:25:39.298663+0000","last_fullsized":"2024-09-22T03:25:39.298663+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":75,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.864449+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:30.864449+0000","last_peered":"2024-09-22T03:25:30.864449+0000","last_clean":"2024-09-22T03:25:30.864449+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:25:30.864449+0000","last_undegraded":"2024-09-22T03:25:30.864449+0000","last_fullsized":"2024-09-22T03:25:30.864449+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"35'356","reported_seq":381,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:36.947999+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:25:36.947999+0000","last_peered":"2024-09-22T03:25:36.947999+0000","last_clean":"2024-09-22T03:25:36.947999+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:25:36.947999+0000","last_undegraded":"2024-09-22T03:25:36.947999+0000","last_fullsized":"2024-09-22T03:25:36.947999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":357,"num_write_kb":2856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.772381+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:30.772381+0000","last_peered":"2024-09-22T03:25:30.772381+0000","last_clean":"2024-09-22T03:25:30.772381+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:25:30.772381+0000","last_undegraded":"2024-09-22T03:25:30.772381+0000","last_fullsized":"2024-09-22T03:25:30.772381+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"35'447","reported_seq":472,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:41.883479+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:25:41.883479+0000","last_peered":"2024-09-22T03:25:41.883479+0000","last_clean":"2024-09-22T03:25:41.883479+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:25:41.883479+0000","last_undegraded":"2024-09-22T03:25:41.883479+0000","last_fullsized":"2024-09-22T03:25:41.883479+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.774266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:30.774266+0000","last_peered":"2024-09-22T03:25:30.774266+0000","last_clean":"2024-09-22T03:25:30.774266+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:25:30.774266+0000","last_undegraded":"2024-09-22T03:25:30.774266+0000","last_fullsized":"2024-09-22T03:25:30.774266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"35'543","reported_seq":568,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:41.670355+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:25:41.670355+0000","last_peered":"2024-09-22T03:25:41.670355+0000","last_clean":"2024-09-22T03:25:41.670355+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:25:41.670355+0000","last_undegraded":"2024-09-22T03:25:41.670355+0000","last_fullsized":"2024-09-22T03:25:41.670355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.771115+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:25:30.771115+0000","last_peered":"2024-09-22T03:25:30.771115+0000","last_clean":"2024-09-22T03:25:30.771115+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:25:30.771115+0000","last_undegraded":"2024-09-22T03:25:30.771115+0000","last_fullsized":"2024-09-22T03:25:30.771115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"35'351","reported_seq":376,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:36.771065+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:25:36.771065+0000","last_peered":"2024-09-22T03:25:36.771065+0000","last_clean":"2024-09-22T03:25:36.771065+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:25:36.771065+0000","last_undegraded":"2024-09-22T03:25:36.771065+0000","last_fullsized":"2024-09-22T03:25:36.771065+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.802359+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:25:30.802359+0000","last_peered":"2024-09-22T03:25:30.802359+0000","last_clean":"2024-09-22T03:25:30.802359+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:25:30.802359+0000","last_undegraded":"2024-09-22T03:25:30.802359+0000","last_fullsized":"2024-09-22T03:25:30.802359+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":56270848,"num_objects":859,"num_object_clones":0,"num_object_copies":2577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6869,"num_write_kb":54952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213845463040,"internally_reserved":0,"allocated":366030848,"data_stored":366030848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6853,"ondisk_log_size":6853,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213845471232,"internally_reserved":0,"allocated":366022656,"data_stored":366022656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53449572352,"internally_reserved":0,"allocated":103301120,"data_stored":103301120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":100736,"kb_used_data":100736,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52196992,"statfs":{"total":53552873472,"available":53449719808,"internally_reserved":0,"allocated":103153664,"data_stored":103153664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":74672,"kb_used_data":74672,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52223056,"statfs":{"total":53552873472,"available":53476409344,"internally_reserved":0,"allocated":76464128,"data_stored":76464128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":85960,"kb_used_data":85960,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52211768,"statfs":{"total":53552873472,"available":53464850432,"internally_reserved":0,"allocated":88023040,"data_stored":88023040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705824,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":95168,"kb_used_data":95168,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52202560,"statfs":{"total":53552873472,"available":53455421440,"internally_reserved":0,"allocated":97452032,"data_stored":97452032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53449572352,"internally_reserved":0,"allocated":103301120,"data_stored":103301120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53455138816,"internally_reserved":0,"allocated":97734656,"data_stored":97734656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53464555520,"internally_reserved":0,"allocated":88317952,"data_stored":88317952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53476204544,"internally_reserved":0,"allocated":76668928,"data_stored":76668928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53449572352,"internally_reserved":0,"allocated":103301120,"data_stored":103301120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53455138816,"internally_reserved":0,"allocated":97734656,"data_stored":97734656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53464547328,"internally_reserved":0,"allocated":88326144,"data_stored":88326144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53476204544,"internally_reserved":0,"allocated":76668928,"data_stored":76668928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53449572352,"internally_reserved":0,"allocated":103301120,"data_stored":103301120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:44.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:25:44.113133+0000 min lat: 0.0119124 max lat: 0.39705 avg lat: 0.0829221 2024-09-22T03:25:44.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:25:44.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 7727 7711 1.50593 1.46094 0.0580112 0.0829221 2024-09-22T03:25:45.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 7924 7908 1.50673 1.53906 0.0697459 0.0828884 2024-09-22T03:25:45.234 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:45.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:45.395+0000 7f4a070ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:45.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:45.402+0000 7f4a070ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:46.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 8114 8098 1.50619 1.48438 0.0816015 0.0828768 2024-09-22T03:25:47.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 15 8307 8292 1.50641 1.51562 0.0433914 0.0829008 2024-09-22T03:25:47.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:47.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:47.761+0000 7fe819f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:47.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:47.768+0000 7fe819f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:48.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 8480 8464 1.50271 1.34375 0.113078 0.0830886 2024-09-22T03:25:48.256 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:48.257 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:48.257 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:25:48.257 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:25:48.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:48.402+0000 7f1eeae6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:48.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:48.413+0000 7f1eeae6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:48.464 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:48.465 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:48.469 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:48.469 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:48.469 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:25:48.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:48.584+0000 7f78f8ea9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:48.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:48.595+0000 7f78f8ea9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:48.685 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:48.685 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:48.685 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:48.685 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:48.685 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:48.686 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:48.686 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:48.686 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:48.686 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:48.686 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:48.686 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:48.686 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:48.688 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:48.689 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:48.689 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:25:48.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:48.837+0000 7fd073d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:48.846+0000 7fd073d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:48.881 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:48.881 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:48.882 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:48.885 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:48.885 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:48.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:25:49.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:49.021+0000 7fb440e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:49.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:49.031+0000 7fb440e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:25:49.066 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:25:49.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:25:49.067 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:25:49.069 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:25:49.069 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:25:49.113 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 8632 8616 1.4957 1.1875 0.146487 0.0834438 2024-09-22T03:25:50.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:50.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 8823 8807 1.49563 1.49219 0.0625341 0.0834873 2024-09-22T03:25:50.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:50.185+0000 7f8bf710e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:50.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:50.192+0000 7f8bf710e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:51.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 16 8981 8965 1.49006 1.23438 0.0798416 0.083768 2024-09-22T03:25:52.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 9152 9136 1.48685 1.33594 0.103704 0.0839828 2024-09-22T03:25:52.404 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:52.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:52.527+0000 7f7b134ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:52.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:52.533+0000 7f7b134ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:53.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 9292 9276 1.47883 1.09375 0.338511 0.0844364 2024-09-22T03:25:54.070 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:54.070 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:25:54.070 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:25:54.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 9484 9468 1.47925 1.5 0.0641729 0.0844102 2024-09-22T03:25:54.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:54.185+0000 7fcb84aba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:54.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:54.192+0000 7fcb84aba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:54.483 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:25:54.484 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:25:54.500 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-22T03:25:53.966591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75309619,"num_objects":1145,"num_object_clones":0,"num_object_copies":3435,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9260,"num_write_kb":74864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9205,"ondisk_log_size":9205,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":460240,"kb_used_data":460240,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208730672,"statfs":{"total":214211493888,"available":213740208128,"internally_reserved":0,"allocated":471285760,"data_stored":471285760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18448384,"num_objects":282,"num_object_clones":0,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2252,"num_write_kb":18016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002145"},"pg_stats":[{"pgid":"4.f","version":"35'599","reported_seq":624,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.887105+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:25:51.887105+0000","last_peered":"2024-09-22T03:25:51.887105+0000","last_clean":"2024-09-22T03:25:51.887105+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:25:51.887105+0000","last_undegraded":"2024-09-22T03:25:51.887105+0000","last_fullsized":"2024-09-22T03:25:51.887105+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"35'503","reported_seq":528,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:49.561958+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:25:49.561958+0000","last_peered":"2024-09-22T03:25:49.561958+0000","last_clean":"2024-09-22T03:25:49.561958+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:25:49.561958+0000","last_undegraded":"2024-09-22T03:25:49.561958+0000","last_fullsized":"2024-09-22T03:25:49.561958+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"35'495","reported_seq":520,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.786112+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:25:51.786112+0000","last_peered":"2024-09-22T03:25:51.786112+0000","last_clean":"2024-09-22T03:25:51.786112+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:25:51.786112+0000","last_undegraded":"2024-09-22T03:25:51.786112+0000","last_fullsized":"2024-09-22T03:25:51.786112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"35'583","reported_seq":608,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.072446+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:25:51.072446+0000","last_peered":"2024-09-22T03:25:51.072446+0000","last_clean":"2024-09-22T03:25:51.072446+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:25:51.072446+0000","last_undegraded":"2024-09-22T03:25:51.072446+0000","last_fullsized":"2024-09-22T03:25:51.072446+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"35'559","reported_seq":584,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.626078+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:25:51.626078+0000","last_peered":"2024-09-22T03:25:51.626078+0000","last_clean":"2024-09-22T03:25:51.626078+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:25:51.626078+0000","last_undegraded":"2024-09-22T03:25:51.626078+0000","last_fullsized":"2024-09-22T03:25:51.626078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"35'535","reported_seq":560,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:49.157508+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:25:49.157508+0000","last_peered":"2024-09-22T03:25:49.157508+0000","last_clean":"2024-09-22T03:25:49.157508+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:25:49.157508+0000","last_undegraded":"2024-09-22T03:25:49.157508+0000","last_fullsized":"2024-09-22T03:25:49.157508+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"35'543","reported_seq":568,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:50.870147+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:25:50.870147+0000","last_peered":"2024-09-22T03:25:50.870147+0000","last_clean":"2024-09-22T03:25:50.870147+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:25:50.870147+0000","last_undegraded":"2024-09-22T03:25:50.870147+0000","last_fullsized":"2024-09-22T03:25:50.870147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"35'663","reported_seq":688,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.665510+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:25:51.665510+0000","last_peered":"2024-09-22T03:25:51.665510+0000","last_clean":"2024-09-22T03:25:51.665510+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:25:51.665510+0000","last_undegraded":"2024-09-22T03:25:51.665510+0000","last_fullsized":"2024-09-22T03:25:51.665510+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"35'623","reported_seq":648,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.703495+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:25:51.703495+0000","last_peered":"2024-09-22T03:25:51.703495+0000","last_clean":"2024-09-22T03:25:51.703495+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:25:51.703495+0000","last_undegraded":"2024-09-22T03:25:51.703495+0000","last_fullsized":"2024-09-22T03:25:51.703495+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":208,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.771791+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:25:30.771791+0000","last_peered":"2024-09-22T03:25:30.771791+0000","last_clean":"2024-09-22T03:25:30.771791+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:25:30.771791+0000","last_undegraded":"2024-09-22T03:25:30.771791+0000","last_fullsized":"2024-09-22T03:25:30.771791+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":74,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.802001+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:25:30.802001+0000","last_peered":"2024-09-22T03:25:30.802001+0000","last_clean":"2024-09-22T03:25:30.802001+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:25:30.802001+0000","last_undegraded":"2024-09-22T03:25:30.802001+0000","last_fullsized":"2024-09-22T03:25:30.802001+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"35'583","reported_seq":608,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.747857+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:25:51.747857+0000","last_peered":"2024-09-22T03:25:51.747857+0000","last_clean":"2024-09-22T03:25:51.747857+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:25:51.747857+0000","last_undegraded":"2024-09-22T03:25:51.747857+0000","last_fullsized":"2024-09-22T03:25:51.747857+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.869396+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:25:30.869396+0000","last_peered":"2024-09-22T03:25:30.869396+0000","last_clean":"2024-09-22T03:25:30.869396+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:25:30.869396+0000","last_undegraded":"2024-09-22T03:25:30.869396+0000","last_fullsized":"2024-09-22T03:25:30.869396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"35'607","reported_seq":632,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.182555+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:25:51.182555+0000","last_peered":"2024-09-22T03:25:51.182555+0000","last_clean":"2024-09-22T03:25:51.182555+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:25:51.182555+0000","last_undegraded":"2024-09-22T03:25:51.182555+0000","last_fullsized":"2024-09-22T03:25:51.182555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.798581+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:30.798581+0000","last_peered":"2024-09-22T03:25:30.798581+0000","last_clean":"2024-09-22T03:25:30.798581+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:25:30.798581+0000","last_undegraded":"2024-09-22T03:25:30.798581+0000","last_fullsized":"2024-09-22T03:25:30.798581+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"35'519","reported_seq":544,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.817828+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:25:51.817828+0000","last_peered":"2024-09-22T03:25:51.817828+0000","last_clean":"2024-09-22T03:25:51.817828+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:25:51.817828+0000","last_undegraded":"2024-09-22T03:25:51.817828+0000","last_fullsized":"2024-09-22T03:25:51.817828+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":75,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.864449+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:25:30.864449+0000","last_peered":"2024-09-22T03:25:30.864449+0000","last_clean":"2024-09-22T03:25:30.864449+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:25:30.864449+0000","last_undegraded":"2024-09-22T03:25:30.864449+0000","last_fullsized":"2024-09-22T03:25:30.864449+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"35'527","reported_seq":552,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:50.508549+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:25:50.508549+0000","last_peered":"2024-09-22T03:25:50.508549+0000","last_clean":"2024-09-22T03:25:50.508549+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:25:50.508549+0000","last_undegraded":"2024-09-22T03:25:50.508549+0000","last_fullsized":"2024-09-22T03:25:50.508549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.772381+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:30.772381+0000","last_peered":"2024-09-22T03:25:30.772381+0000","last_clean":"2024-09-22T03:25:30.772381+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:25:30.772381+0000","last_undegraded":"2024-09-22T03:25:30.772381+0000","last_fullsized":"2024-09-22T03:25:30.772381+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"35'584","reported_seq":609,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.524682+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:25:51.524682+0000","last_peered":"2024-09-22T03:25:51.524682+0000","last_clean":"2024-09-22T03:25:51.524682+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:25:51.524682+0000","last_undegraded":"2024-09-22T03:25:51.524682+0000","last_fullsized":"2024-09-22T03:25:51.524682+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4792320,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":585,"num_write_kb":4680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.774266+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:25:30.774266+0000","last_peered":"2024-09-22T03:25:30.774266+0000","last_clean":"2024-09-22T03:25:30.774266+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:25:30.774266+0000","last_undegraded":"2024-09-22T03:25:30.774266+0000","last_fullsized":"2024-09-22T03:25:30.774266+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"35'695","reported_seq":720,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.580364+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:25:51.580364+0000","last_peered":"2024-09-22T03:25:51.580364+0000","last_clean":"2024-09-22T03:25:51.580364+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:25:51.580364+0000","last_undegraded":"2024-09-22T03:25:51.580364+0000","last_fullsized":"2024-09-22T03:25:51.580364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.771115+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:25:30.771115+0000","last_peered":"2024-09-22T03:25:30.771115+0000","last_clean":"2024-09-22T03:25:30.771115+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:25:30.771115+0000","last_undegraded":"2024-09-22T03:25:30.771115+0000","last_fullsized":"2024-09-22T03:25:30.771115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"35'487","reported_seq":512,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:51.141056+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:25:51.141056+0000","last_peered":"2024-09-22T03:25:51.141056+0000","last_clean":"2024-09-22T03:25:51.141056+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:25:51.141056+0000","last_undegraded":"2024-09-22T03:25:51.141056+0000","last_fullsized":"2024-09-22T03:25:51.141056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-22T03:25:30.802359+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:25:30.802359+0000","last_peered":"2024-09-22T03:25:30.802359+0000","last_clean":"2024-09-22T03:25:30.802359+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:25:30.802359+0000","last_undegraded":"2024-09-22T03:25:30.802359+0000","last_fullsized":"2024-09-22T03:25:30.802359+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":74719232,"num_objects":1141,"num_object_clones":0,"num_object_copies":3423,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9121,"num_write_kb":72968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213739872256,"internally_reserved":0,"allocated":471621632,"data_stored":471621632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9105,"ondisk_log_size":9105,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213739872256,"internally_reserved":0,"allocated":471621632,"data_stored":471621632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53424115712,"internally_reserved":0,"allocated":128757760,"data_stored":128757760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":125708,"kb_used_data":125708,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52172020,"statfs":{"total":53552873472,"available":53424148480,"internally_reserved":0,"allocated":128724992,"data_stored":128724992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":93392,"kb_used_data":93392,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52204336,"statfs":{"total":53552873472,"available":53457240064,"internally_reserved":0,"allocated":95633408,"data_stored":95633408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705834,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":122572,"kb_used_data":122572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52175156,"statfs":{"total":53552873472,"available":53427359744,"internally_reserved":0,"allocated":125513728,"data_stored":125513728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705834,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":118568,"kb_used_data":118568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52179160,"statfs":{"total":53552873472,"available":53431459840,"internally_reserved":0,"allocated":121413632,"data_stored":121413632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53424115712,"internally_reserved":0,"allocated":128757760,"data_stored":128757760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53431369728,"internally_reserved":0,"allocated":121503744,"data_stored":121503744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53427286016,"internally_reserved":0,"allocated":125587456,"data_stored":125587456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53457100800,"internally_reserved":0,"allocated":95772672,"data_stored":95772672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53424115712,"internally_reserved":0,"allocated":128757760,"data_stored":128757760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53431369728,"internally_reserved":0,"allocated":121503744,"data_stored":121503744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53427286016,"internally_reserved":0,"allocated":125587456,"data_stored":125587456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53457100800,"internally_reserved":0,"allocated":95772672,"data_stored":95772672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53424115712,"internally_reserved":0,"allocated":128757760,"data_stored":128757760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:25:54.715 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:54.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:54.841+0000 7f4809f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:54.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:54.848+0000 7f4809f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:55.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 9669 9653 1.47858 1.44531 0.0714235 0.0844718 2024-09-22T03:25:56.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 9824 9808 1.47343 1.21094 0.105884 0.0847402 2024-09-22T03:25:57.064 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:25:57.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 9985 9969 1.46936 1.25781 0.0872262 0.0849617 2024-09-22T03:25:57.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:57.221+0000 7fcc55d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:57.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:57.228+0000 7fcc55d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:58.114 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 10177 10161 1.46993 1.5 0.0546449 0.084967 2024-09-22T03:25:59.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 10363 10347 1.46962 1.45312 0.052303 0.0849582 2024-09-22T03:25:59.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:25:59.502 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:25:59.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:25:59.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:59.609+0000 7f707982c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:59.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:59.615+0000 7f707982c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:59.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:59.687+0000 7f8150a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:25:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:25:59.697+0000 7f8150a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:00.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 10525 10509 1.46597 1.26562 0.0891523 0.0851373 2024-09-22T03:26:00.720 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:26:00.740 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:26:00.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:00.851+0000 7f5097197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:00.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:00.861+0000 7f5097197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:01.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 10706 10690 1.46506 1.41406 0.0919194 0.0852451 2024-09-22T03:26:01.720 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:26:01.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:26:01.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:01.901+0000 7f9990597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:01.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:01.917+0000 7f9990597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:01.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:01.991+0000 7f0642cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:01.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:01.997+0000 7f0642cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:02.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 10856 10840 1.46 1.17188 0.134822 0.0855694 2024-09-22T03:26:02.726 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:26:02.749 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:26:02.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:02.866+0000 7fb87e997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:02.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:02.880+0000 7fb87e997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:03.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 11011 10995 1.45578 1.21094 0.108881 0.0857772 2024-09-22T03:26:03.730 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:26:03.748 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:26:03.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:03.907+0000 7f01d7ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:03.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:03.921+0000 7f01d7ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:26:04.114872+0000 min lat: 0.0119124 max lat: 0.39705 avg lat: 0.0858189 2024-09-22T03:26:04.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:26:04.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 11193 11177 1.45521 1.42188 0.0617408 0.0858189 2024-09-22T03:26:04.180 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:04.180 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:04.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:04.199 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-22T03:26:03.968312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89309747,"num_objects":1358,"num_object_clones":0,"num_object_copies":4074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10969,"num_write_kb":88536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10925,"ondisk_log_size":10925,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":545152,"kb_used_data":545152,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208645760,"statfs":{"total":214211493888,"available":213653258240,"internally_reserved":0,"allocated":558235648,"data_stored":558235648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7168000,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":875,"num_write_kb":7000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000713"},"pg_stats":[{"pgid":"4.f","version":"37'775","reported_seq":806,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.790034+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:01.790034+0000","last_peered":"2024-09-22T03:26:01.790034+0000","last_clean":"2024-09-22T03:26:01.790034+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:01.790034+0000","last_undegraded":"2024-09-22T03:26:01.790034+0000","last_fullsized":"2024-09-22T03:26:01.790034+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"36'576","reported_seq":606,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.736980+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:01.736980+0000","last_peered":"2024-09-22T03:26:01.736980+0000","last_clean":"2024-09-22T03:26:01.736980+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:01.736980+0000","last_undegraded":"2024-09-22T03:26:01.736980+0000","last_fullsized":"2024-09-22T03:26:01.736980+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"37'615","reported_seq":646,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.845929+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:01.845929+0000","last_peered":"2024-09-22T03:26:01.845929+0000","last_clean":"2024-09-22T03:26:01.845929+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:01.845929+0000","last_undegraded":"2024-09-22T03:26:01.845929+0000","last_fullsized":"2024-09-22T03:26:01.845929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"36'656","reported_seq":686,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751844+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:01.751844+0000","last_peered":"2024-09-22T03:26:01.751844+0000","last_clean":"2024-09-22T03:26:01.751844+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:01.751844+0000","last_undegraded":"2024-09-22T03:26:01.751844+0000","last_fullsized":"2024-09-22T03:26:01.751844+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"36'696","reported_seq":726,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751509+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:01.751509+0000","last_peered":"2024-09-22T03:26:01.751509+0000","last_clean":"2024-09-22T03:26:01.751509+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:01.751509+0000","last_undegraded":"2024-09-22T03:26:01.751509+0000","last_fullsized":"2024-09-22T03:26:01.751509+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"35'632","reported_seq":662,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.748137+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:01.748137+0000","last_peered":"2024-09-22T03:26:01.748137+0000","last_clean":"2024-09-22T03:26:01.748137+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:01.748137+0000","last_undegraded":"2024-09-22T03:26:01.748137+0000","last_fullsized":"2024-09-22T03:26:01.748137+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"37'653","reported_seq":684,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.940961+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:01.940961+0000","last_peered":"2024-09-22T03:26:01.940961+0000","last_clean":"2024-09-22T03:26:01.940961+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:01.940961+0000","last_undegraded":"2024-09-22T03:26:01.940961+0000","last_fullsized":"2024-09-22T03:26:01.940961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":654,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"37'751","reported_seq":782,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.810307+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:01.810307+0000","last_peered":"2024-09-22T03:26:01.810307+0000","last_clean":"2024-09-22T03:26:01.810307+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:01.810307+0000","last_undegraded":"2024-09-22T03:26:01.810307+0000","last_fullsized":"2024-09-22T03:26:01.810307+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"35'712","reported_seq":742,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.743969+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:01.743969+0000","last_peered":"2024-09-22T03:26:01.743969+0000","last_clean":"2024-09-22T03:26:01.743969+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:01.743969+0000","last_undegraded":"2024-09-22T03:26:01.743969+0000","last_fullsized":"2024-09-22T03:26:01.743969+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":214,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.749653+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:01.749653+0000","last_peered":"2024-09-22T03:26:01.749653+0000","last_clean":"2024-09-22T03:26:01.749653+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:01.749653+0000","last_undegraded":"2024-09-22T03:26:01.749653+0000","last_fullsized":"2024-09-22T03:26:01.749653+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":80,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.749545+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:01.749545+0000","last_peered":"2024-09-22T03:26:01.749545+0000","last_clean":"2024-09-22T03:26:01.749545+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:01.749545+0000","last_undegraded":"2024-09-22T03:26:01.749545+0000","last_fullsized":"2024-09-22T03:26:01.749545+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"36'696","reported_seq":726,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.750924+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:01.750924+0000","last_peered":"2024-09-22T03:26:01.750924+0000","last_clean":"2024-09-22T03:26:01.750924+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:01.750924+0000","last_undegraded":"2024-09-22T03:26:01.750924+0000","last_fullsized":"2024-09-22T03:26:01.750924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.749809+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:01.749809+0000","last_peered":"2024-09-22T03:26:01.749809+0000","last_clean":"2024-09-22T03:26:01.749809+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:01.749809+0000","last_undegraded":"2024-09-22T03:26:01.749809+0000","last_fullsized":"2024-09-22T03:26:01.749809+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"35'704","reported_seq":734,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.736983+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:01.736983+0000","last_peered":"2024-09-22T03:26:01.736983+0000","last_clean":"2024-09-22T03:26:01.736983+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:01.736983+0000","last_undegraded":"2024-09-22T03:26:01.736983+0000","last_fullsized":"2024-09-22T03:26:01.736983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.744815+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:01.744815+0000","last_peered":"2024-09-22T03:26:01.744815+0000","last_clean":"2024-09-22T03:26:01.744815+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:01.744815+0000","last_undegraded":"2024-09-22T03:26:01.744815+0000","last_fullsized":"2024-09-22T03:26:01.744815+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"36'600","reported_seq":630,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.733550+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:01.733550+0000","last_peered":"2024-09-22T03:26:01.733550+0000","last_clean":"2024-09-22T03:26:01.733550+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:01.733550+0000","last_undegraded":"2024-09-22T03:26:01.733550+0000","last_fullsized":"2024-09-22T03:26:01.733550+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":81,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.745462+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:01.745462+0000","last_peered":"2024-09-22T03:26:01.745462+0000","last_clean":"2024-09-22T03:26:01.745462+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:01.745462+0000","last_undegraded":"2024-09-22T03:26:01.745462+0000","last_fullsized":"2024-09-22T03:26:01.745462+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"37'631","reported_seq":662,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.944739+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:01.944739+0000","last_peered":"2024-09-22T03:26:01.944739+0000","last_clean":"2024-09-22T03:26:01.944739+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:01.944739+0000","last_undegraded":"2024-09-22T03:26:01.944739+0000","last_fullsized":"2024-09-22T03:26:01.944739+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.737408+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:01.737408+0000","last_peered":"2024-09-22T03:26:01.737408+0000","last_clean":"2024-09-22T03:26:01.737408+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:01.737408+0000","last_undegraded":"2024-09-22T03:26:01.737408+0000","last_fullsized":"2024-09-22T03:26:01.737408+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"35'672","reported_seq":702,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.748908+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:01.748908+0000","last_peered":"2024-09-22T03:26:01.748908+0000","last_clean":"2024-09-22T03:26:01.748908+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:01.748908+0000","last_undegraded":"2024-09-22T03:26:01.748908+0000","last_fullsized":"2024-09-22T03:26:01.748908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751920+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:01.751920+0000","last_peered":"2024-09-22T03:26:01.751920+0000","last_clean":"2024-09-22T03:26:01.751920+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:01.751920+0000","last_undegraded":"2024-09-22T03:26:01.751920+0000","last_fullsized":"2024-09-22T03:26:01.751920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"35'832","reported_seq":862,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751102+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:01.751102+0000","last_peered":"2024-09-22T03:26:01.751102+0000","last_clean":"2024-09-22T03:26:01.751102+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:01.751102+0000","last_undegraded":"2024-09-22T03:26:01.751102+0000","last_fullsized":"2024-09-22T03:26:01.751102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.737279+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:01.737279+0000","last_peered":"2024-09-22T03:26:01.737279+0000","last_clean":"2024-09-22T03:26:01.737279+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:01.737279+0000","last_undegraded":"2024-09-22T03:26:01.737279+0000","last_fullsized":"2024-09-22T03:26:01.737279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"36'624","reported_seq":654,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.748293+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:01.748293+0000","last_peered":"2024-09-22T03:26:01.748293+0000","last_clean":"2024-09-22T03:26:01.748293+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:01.748293+0000","last_undegraded":"2024-09-22T03:26:01.748293+0000","last_fullsized":"2024-09-22T03:26:01.748293+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.747516+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:01.747516+0000","last_peered":"2024-09-22T03:26:01.747516+0000","last_clean":"2024-09-22T03:26:01.747516+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:01.747516+0000","last_undegraded":"2024-09-22T03:26:01.747516+0000","last_fullsized":"2024-09-22T03:26:01.747516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":88719360,"num_objects":1354,"num_object_clones":0,"num_object_copies":4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":86640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213652955136,"internally_reserved":0,"allocated":558538752,"data_stored":558538752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10825,"ondisk_log_size":10825,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213652955136,"internally_reserved":0,"allocated":558538752,"data_stored":558538752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53400928256,"internally_reserved":0,"allocated":151945216,"data_stored":151945216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":148248,"kb_used_data":148248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52149480,"statfs":{"total":53552873472,"available":53401067520,"internally_reserved":0,"allocated":151805952,"data_stored":151805952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":111408,"kb_used_data":111408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52186320,"statfs":{"total":53552873472,"available":53438791680,"internally_reserved":0,"allocated":114081792,"data_stored":114081792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":144500,"kb_used_data":144500,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52153228,"statfs":{"total":53552873472,"available":53404905472,"internally_reserved":0,"allocated":147968000,"data_stored":147968000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":140996,"kb_used_data":140996,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52156732,"statfs":{"total":53552873472,"available":53408493568,"internally_reserved":0,"allocated":144379904,"data_stored":144379904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53400928256,"internally_reserved":0,"allocated":151945216,"data_stored":151945216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53408403456,"internally_reserved":0,"allocated":144470016,"data_stored":144470016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53404889088,"internally_reserved":0,"allocated":147984384,"data_stored":147984384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53438742528,"internally_reserved":0,"allocated":114130944,"data_stored":114130944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53400920064,"internally_reserved":0,"allocated":151953408,"data_stored":151953408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53408403456,"internally_reserved":0,"allocated":144470016,"data_stored":144470016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53404889088,"internally_reserved":0,"allocated":147984384,"data_stored":147984384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53438742528,"internally_reserved":0,"allocated":114130944,"data_stored":114130944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53400920064,"internally_reserved":0,"allocated":151953408,"data_stored":151953408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:04.200 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.306+0000 7fc5b7117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.312+0000 7fc5b7117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.350+0000 7f58ccf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.359+0000 7f58ccf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.558 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:04.558 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:04.575 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-22T03:26:03.968312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89309747,"num_objects":1358,"num_object_clones":0,"num_object_copies":4074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10969,"num_write_kb":88536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10925,"ondisk_log_size":10925,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":545152,"kb_used_data":545152,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208645760,"statfs":{"total":214211493888,"available":213653258240,"internally_reserved":0,"allocated":558235648,"data_stored":558235648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7168000,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":875,"num_write_kb":7000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000713"},"pg_stats":[{"pgid":"4.f","version":"37'775","reported_seq":806,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.790034+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:01.790034+0000","last_peered":"2024-09-22T03:26:01.790034+0000","last_clean":"2024-09-22T03:26:01.790034+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:01.790034+0000","last_undegraded":"2024-09-22T03:26:01.790034+0000","last_fullsized":"2024-09-22T03:26:01.790034+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"36'576","reported_seq":606,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.736980+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:01.736980+0000","last_peered":"2024-09-22T03:26:01.736980+0000","last_clean":"2024-09-22T03:26:01.736980+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:01.736980+0000","last_undegraded":"2024-09-22T03:26:01.736980+0000","last_fullsized":"2024-09-22T03:26:01.736980+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"37'615","reported_seq":646,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.845929+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:01.845929+0000","last_peered":"2024-09-22T03:26:01.845929+0000","last_clean":"2024-09-22T03:26:01.845929+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:01.845929+0000","last_undegraded":"2024-09-22T03:26:01.845929+0000","last_fullsized":"2024-09-22T03:26:01.845929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"36'656","reported_seq":686,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751844+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:01.751844+0000","last_peered":"2024-09-22T03:26:01.751844+0000","last_clean":"2024-09-22T03:26:01.751844+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:01.751844+0000","last_undegraded":"2024-09-22T03:26:01.751844+0000","last_fullsized":"2024-09-22T03:26:01.751844+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"36'696","reported_seq":726,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751509+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:01.751509+0000","last_peered":"2024-09-22T03:26:01.751509+0000","last_clean":"2024-09-22T03:26:01.751509+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:01.751509+0000","last_undegraded":"2024-09-22T03:26:01.751509+0000","last_fullsized":"2024-09-22T03:26:01.751509+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"35'632","reported_seq":662,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.748137+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:01.748137+0000","last_peered":"2024-09-22T03:26:01.748137+0000","last_clean":"2024-09-22T03:26:01.748137+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:01.748137+0000","last_undegraded":"2024-09-22T03:26:01.748137+0000","last_fullsized":"2024-09-22T03:26:01.748137+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"37'653","reported_seq":684,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.940961+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:01.940961+0000","last_peered":"2024-09-22T03:26:01.940961+0000","last_clean":"2024-09-22T03:26:01.940961+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:01.940961+0000","last_undegraded":"2024-09-22T03:26:01.940961+0000","last_fullsized":"2024-09-22T03:26:01.940961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":654,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"37'751","reported_seq":782,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.810307+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:01.810307+0000","last_peered":"2024-09-22T03:26:01.810307+0000","last_clean":"2024-09-22T03:26:01.810307+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:01.810307+0000","last_undegraded":"2024-09-22T03:26:01.810307+0000","last_fullsized":"2024-09-22T03:26:01.810307+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"35'712","reported_seq":742,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.743969+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:01.743969+0000","last_peered":"2024-09-22T03:26:01.743969+0000","last_clean":"2024-09-22T03:26:01.743969+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:01.743969+0000","last_undegraded":"2024-09-22T03:26:01.743969+0000","last_fullsized":"2024-09-22T03:26:01.743969+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":214,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.749653+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:01.749653+0000","last_peered":"2024-09-22T03:26:01.749653+0000","last_clean":"2024-09-22T03:26:01.749653+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:01.749653+0000","last_undegraded":"2024-09-22T03:26:01.749653+0000","last_fullsized":"2024-09-22T03:26:01.749653+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":80,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.749545+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:01.749545+0000","last_peered":"2024-09-22T03:26:01.749545+0000","last_clean":"2024-09-22T03:26:01.749545+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:01.749545+0000","last_undegraded":"2024-09-22T03:26:01.749545+0000","last_fullsized":"2024-09-22T03:26:01.749545+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"36'696","reported_seq":726,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.750924+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:01.750924+0000","last_peered":"2024-09-22T03:26:01.750924+0000","last_clean":"2024-09-22T03:26:01.750924+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:01.750924+0000","last_undegraded":"2024-09-22T03:26:01.750924+0000","last_fullsized":"2024-09-22T03:26:01.750924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.749809+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:01.749809+0000","last_peered":"2024-09-22T03:26:01.749809+0000","last_clean":"2024-09-22T03:26:01.749809+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:01.749809+0000","last_undegraded":"2024-09-22T03:26:01.749809+0000","last_fullsized":"2024-09-22T03:26:01.749809+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"35'704","reported_seq":734,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.736983+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:01.736983+0000","last_peered":"2024-09-22T03:26:01.736983+0000","last_clean":"2024-09-22T03:26:01.736983+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:01.736983+0000","last_undegraded":"2024-09-22T03:26:01.736983+0000","last_fullsized":"2024-09-22T03:26:01.736983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.744815+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:01.744815+0000","last_peered":"2024-09-22T03:26:01.744815+0000","last_clean":"2024-09-22T03:26:01.744815+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:01.744815+0000","last_undegraded":"2024-09-22T03:26:01.744815+0000","last_fullsized":"2024-09-22T03:26:01.744815+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"36'600","reported_seq":630,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.733550+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:01.733550+0000","last_peered":"2024-09-22T03:26:01.733550+0000","last_clean":"2024-09-22T03:26:01.733550+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:01.733550+0000","last_undegraded":"2024-09-22T03:26:01.733550+0000","last_fullsized":"2024-09-22T03:26:01.733550+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":81,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.745462+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:01.745462+0000","last_peered":"2024-09-22T03:26:01.745462+0000","last_clean":"2024-09-22T03:26:01.745462+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:01.745462+0000","last_undegraded":"2024-09-22T03:26:01.745462+0000","last_fullsized":"2024-09-22T03:26:01.745462+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"37'631","reported_seq":662,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.944739+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:01.944739+0000","last_peered":"2024-09-22T03:26:01.944739+0000","last_clean":"2024-09-22T03:26:01.944739+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:01.944739+0000","last_undegraded":"2024-09-22T03:26:01.944739+0000","last_fullsized":"2024-09-22T03:26:01.944739+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.737408+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:01.737408+0000","last_peered":"2024-09-22T03:26:01.737408+0000","last_clean":"2024-09-22T03:26:01.737408+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:01.737408+0000","last_undegraded":"2024-09-22T03:26:01.737408+0000","last_fullsized":"2024-09-22T03:26:01.737408+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"35'672","reported_seq":702,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.748908+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:01.748908+0000","last_peered":"2024-09-22T03:26:01.748908+0000","last_clean":"2024-09-22T03:26:01.748908+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:01.748908+0000","last_undegraded":"2024-09-22T03:26:01.748908+0000","last_fullsized":"2024-09-22T03:26:01.748908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751920+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:01.751920+0000","last_peered":"2024-09-22T03:26:01.751920+0000","last_clean":"2024-09-22T03:26:01.751920+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:01.751920+0000","last_undegraded":"2024-09-22T03:26:01.751920+0000","last_fullsized":"2024-09-22T03:26:01.751920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"35'832","reported_seq":862,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.751102+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:01.751102+0000","last_peered":"2024-09-22T03:26:01.751102+0000","last_clean":"2024-09-22T03:26:01.751102+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:01.751102+0000","last_undegraded":"2024-09-22T03:26:01.751102+0000","last_fullsized":"2024-09-22T03:26:01.751102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.737279+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:01.737279+0000","last_peered":"2024-09-22T03:26:01.737279+0000","last_clean":"2024-09-22T03:26:01.737279+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:01.737279+0000","last_undegraded":"2024-09-22T03:26:01.737279+0000","last_fullsized":"2024-09-22T03:26:01.737279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"36'624","reported_seq":654,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.748293+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:01.748293+0000","last_peered":"2024-09-22T03:26:01.748293+0000","last_clean":"2024-09-22T03:26:01.748293+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:01.748293+0000","last_undegraded":"2024-09-22T03:26:01.748293+0000","last_fullsized":"2024-09-22T03:26:01.748293+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-22T03:26:01.747516+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:01.747516+0000","last_peered":"2024-09-22T03:26:01.747516+0000","last_clean":"2024-09-22T03:26:01.747516+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:01.747516+0000","last_undegraded":"2024-09-22T03:26:01.747516+0000","last_fullsized":"2024-09-22T03:26:01.747516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":88719360,"num_objects":1354,"num_object_clones":0,"num_object_copies":4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":86640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213652955136,"internally_reserved":0,"allocated":558538752,"data_stored":558538752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10825,"ondisk_log_size":10825,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213652955136,"internally_reserved":0,"allocated":558538752,"data_stored":558538752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53400928256,"internally_reserved":0,"allocated":151945216,"data_stored":151945216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":148248,"kb_used_data":148248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52149480,"statfs":{"total":53552873472,"available":53401067520,"internally_reserved":0,"allocated":151805952,"data_stored":151805952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":111408,"kb_used_data":111408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52186320,"statfs":{"total":53552873472,"available":53438791680,"internally_reserved":0,"allocated":114081792,"data_stored":114081792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":144500,"kb_used_data":144500,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52153228,"statfs":{"total":53552873472,"available":53404905472,"internally_reserved":0,"allocated":147968000,"data_stored":147968000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":140996,"kb_used_data":140996,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52156732,"statfs":{"total":53552873472,"available":53408493568,"internally_reserved":0,"allocated":144379904,"data_stored":144379904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53400928256,"internally_reserved":0,"allocated":151945216,"data_stored":151945216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53408403456,"internally_reserved":0,"allocated":144470016,"data_stored":144470016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53404889088,"internally_reserved":0,"allocated":147984384,"data_stored":147984384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53438742528,"internally_reserved":0,"allocated":114130944,"data_stored":114130944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53400920064,"internally_reserved":0,"allocated":151953408,"data_stored":151953408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53408403456,"internally_reserved":0,"allocated":144470016,"data_stored":144470016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53404889088,"internally_reserved":0,"allocated":147984384,"data_stored":147984384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53438742528,"internally_reserved":0,"allocated":114130944,"data_stored":114130944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53400920064,"internally_reserved":0,"allocated":151953408,"data_stored":151953408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:04.576 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:26:04.576 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:04.577 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:04.577 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:04.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.733+0000 7fb02c8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.744+0000 7fb02c8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.795 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:04.795 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:04.795 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:04.795 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:04.796 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:04.798 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:04.799 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:04.799 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:04.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.922+0000 7f0247cba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:04.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:04.932+0000 7f0247cba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:05.028 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:05.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:05.031 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:05.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:05.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 11334 11318 1.44941 1.10156 0.171795 0.0861169 2024-09-22T03:26:05.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:05.162+0000 7ff2526a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:05.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:05.175+0000 7ff2526a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:05.202 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:05.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:05.205 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:05.205 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:05.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:05.321+0000 7f42a2557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:05.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:05.331+0000 7f42a2557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:05.353 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:05.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:05.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:05.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:05.354 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:05.357 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:05.357 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:06.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 11504 11488 1.44746 1.32812 0.04729 0.0862573 2024-09-22T03:26:06.556 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:06.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:06.679+0000 7f3f26a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:06.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:06.685+0000 7f3f26a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:07.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 11696 11680 1.44829 1.5 0.139142 0.0862091 2024-09-22T03:26:08.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 16 11876 11860 1.44763 1.40625 0.0460413 0.0862514 2024-09-22T03:26:08.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:09.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:09.100+0000 7fec8cd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:09.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:09.106+0000 7fec8cd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:09.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 12056 12040 1.44699 1.40625 0.0646048 0.0863064 2024-09-22T03:26:10.115 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 12247 12231 1.44767 1.49219 0.0512476 0.0862534 2024-09-22T03:26:10.358 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:10.358 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:10.358 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:10.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:10.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:10.510+0000 7fd3af0ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:10.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:10.523+0000 7fd3af0ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:10.609 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:10.609 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:10.609 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:10.609 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:10.609 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:10.609 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:10.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:10.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:10.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:10.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:10.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:10.610 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:10.612 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:10.613 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:10.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:10.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:10.735+0000 7f121c6a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:10.747 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:10.746+0000 7f121c6a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:10.848 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:10.848 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:10.848 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:10.849 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:10.851 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:10.852 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:10.852 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:10.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:10.967+0000 7ff43e6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:10.979+0000 7ff43e6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:11.040 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:11.040 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:11.041 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:11.043 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:11.044 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:11.044 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:11.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 12415 12399 1.44565 1.3125 0.0402362 0.0863519 2024-09-22T03:26:11.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:11.180+0000 7fa0f466c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:11.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:11.194+0000 7fa0f466c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:11.220 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:11.221 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:11.221 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:11.221 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:11.224 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:11.224 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:11.350 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:11.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:11.471+0000 7fed61b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:11.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:11.476+0000 7fed61b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:12.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 12610 12594 1.4468 1.52344 0.0701834 0.0863258 2024-09-22T03:26:13.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 12744 12728 1.441 1.04688 0.0960098 0.0865876 2024-09-22T03:26:13.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:13.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:13.834+0000 7fe34184f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:13.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:13.840+0000 7fe34184f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:14.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 12915 12899 1.4395 1.33594 0.0769726 0.0867412 2024-09-22T03:26:15.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 13070 13054 1.43628 1.21094 0.0427721 0.0869761 2024-09-22T03:26:16.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:16.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 13194 13178 1.42978 0.96875 0.100901 0.0873644 2024-09-22T03:26:16.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:16.194+0000 7f0ea822c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:16.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:16.199+0000 7f0ea822c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:16.225 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:16.225 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:16.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:16.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:16.351+0000 7f2b16717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:16.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:16.359+0000 7f2b16717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:16.568 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:16.569 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:16.583 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-22T03:26:15.970545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103621171,"num_objects":1577,"num_object_clones":0,"num_object_copies":4731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12716,"num_write_kb":102512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12661,"ondisk_log_size":12661,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":632444,"kb_used_data":632444,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208558468,"statfs":{"total":214211493888,"available":213563871232,"internally_reserved":0,"allocated":647622656,"data_stored":647622656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14311424,"num_objects":219,"num_object_clones":0,"num_object_copies":657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1747,"num_write_kb":13976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002233"},"pg_stats":[{"pgid":"4.f","version":"39'879","reported_seq":916,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.580346+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:11.580346+0000","last_peered":"2024-09-22T03:26:11.580346+0000","last_clean":"2024-09-22T03:26:11.580346+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:11.580346+0000","last_undegraded":"2024-09-22T03:26:11.580346+0000","last_fullsized":"2024-09-22T03:26:11.580346+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"39'695","reported_seq":732,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.932654+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:11.932654+0000","last_peered":"2024-09-22T03:26:11.932654+0000","last_clean":"2024-09-22T03:26:11.932654+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:11.932654+0000","last_undegraded":"2024-09-22T03:26:11.932654+0000","last_fullsized":"2024-09-22T03:26:11.932654+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"39'751","reported_seq":788,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.237311+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:11.237311+0000","last_peered":"2024-09-22T03:26:11.237311+0000","last_clean":"2024-09-22T03:26:11.237311+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:11.237311+0000","last_undegraded":"2024-09-22T03:26:11.237311+0000","last_fullsized":"2024-09-22T03:26:11.237311+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"39'735","reported_seq":772,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:10.799368+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:10.799368+0000","last_peered":"2024-09-22T03:26:10.799368+0000","last_clean":"2024-09-22T03:26:10.799368+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:10.799368+0000","last_undegraded":"2024-09-22T03:26:10.799368+0000","last_fullsized":"2024-09-22T03:26:10.799368+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"39'831","reported_seq":868,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:09.550461+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:09.550461+0000","last_peered":"2024-09-22T03:26:09.550461+0000","last_clean":"2024-09-22T03:26:09.550461+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:09.550461+0000","last_undegraded":"2024-09-22T03:26:09.550461+0000","last_fullsized":"2024-09-22T03:26:09.550461+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"39'751","reported_seq":788,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.595595+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:11.595595+0000","last_peered":"2024-09-22T03:26:11.595595+0000","last_clean":"2024-09-22T03:26:11.595595+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:11.595595+0000","last_undegraded":"2024-09-22T03:26:11.595595+0000","last_fullsized":"2024-09-22T03:26:11.595595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"39'720","reported_seq":757,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.103811+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:11.103811+0000","last_peered":"2024-09-22T03:26:11.103811+0000","last_clean":"2024-09-22T03:26:11.103811+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:11.103811+0000","last_undegraded":"2024-09-22T03:26:11.103811+0000","last_fullsized":"2024-09-22T03:26:11.103811+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"39'839","reported_seq":876,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.717790+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:11.717790+0000","last_peered":"2024-09-22T03:26:11.717790+0000","last_clean":"2024-09-22T03:26:11.717790+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:11.717790+0000","last_undegraded":"2024-09-22T03:26:11.717790+0000","last_fullsized":"2024-09-22T03:26:11.717790+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881280,"num_objects":105,"num_object_clones":0,"num_object_copies":315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"39'863","reported_seq":900,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:09.962854+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:09.962854+0000","last_peered":"2024-09-22T03:26:09.962854+0000","last_clean":"2024-09-22T03:26:09.962854+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:09.962854+0000","last_undegraded":"2024-09-22T03:26:09.962854+0000","last_fullsized":"2024-09-22T03:26:09.962854+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":220,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.746474+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:03.746474+0000","last_peered":"2024-09-22T03:26:03.746474+0000","last_clean":"2024-09-22T03:26:03.746474+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:03.746474+0000","last_undegraded":"2024-09-22T03:26:03.746474+0000","last_fullsized":"2024-09-22T03:26:03.746474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":86,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.771787+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:03.771787+0000","last_peered":"2024-09-22T03:26:03.771787+0000","last_clean":"2024-09-22T03:26:03.771787+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:03.771787+0000","last_undegraded":"2024-09-22T03:26:03.771787+0000","last_fullsized":"2024-09-22T03:26:03.771787+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"39'799","reported_seq":836,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.824276+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:11.824276+0000","last_peered":"2024-09-22T03:26:11.824276+0000","last_clean":"2024-09-22T03:26:11.824276+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:11.824276+0000","last_undegraded":"2024-09-22T03:26:11.824276+0000","last_fullsized":"2024-09-22T03:26:11.824276+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.754426+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:03.754426+0000","last_peered":"2024-09-22T03:26:03.754426+0000","last_clean":"2024-09-22T03:26:03.754426+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:03.754426+0000","last_undegraded":"2024-09-22T03:26:03.754426+0000","last_fullsized":"2024-09-22T03:26:03.754426+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"39'831","reported_seq":868,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.916172+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:11.916172+0000","last_peered":"2024-09-22T03:26:11.916172+0000","last_clean":"2024-09-22T03:26:11.916172+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:11.916172+0000","last_undegraded":"2024-09-22T03:26:11.916172+0000","last_fullsized":"2024-09-22T03:26:11.916172+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":85,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.766028+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:03.766028+0000","last_peered":"2024-09-22T03:26:03.766028+0000","last_clean":"2024-09-22T03:26:03.766028+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:03.766028+0000","last_undegraded":"2024-09-22T03:26:03.766028+0000","last_fullsized":"2024-09-22T03:26:03.766028+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"39'703","reported_seq":740,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:10.754235+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:10.754235+0000","last_peered":"2024-09-22T03:26:10.754235+0000","last_clean":"2024-09-22T03:26:10.754235+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:10.754235+0000","last_undegraded":"2024-09-22T03:26:10.754235+0000","last_fullsized":"2024-09-22T03:26:10.754235+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.749829+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:03.749829+0000","last_peered":"2024-09-22T03:26:03.749829+0000","last_clean":"2024-09-22T03:26:03.749829+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:03.749829+0000","last_undegraded":"2024-09-22T03:26:03.749829+0000","last_fullsized":"2024-09-22T03:26:03.749829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"39'735","reported_seq":772,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.620393+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:11.620393+0000","last_peered":"2024-09-22T03:26:11.620393+0000","last_clean":"2024-09-22T03:26:11.620393+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:11.620393+0000","last_undegraded":"2024-09-22T03:26:11.620393+0000","last_fullsized":"2024-09-22T03:26:11.620393+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.759451+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:03.759451+0000","last_peered":"2024-09-22T03:26:03.759451+0000","last_clean":"2024-09-22T03:26:03.759451+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:03.759451+0000","last_undegraded":"2024-09-22T03:26:03.759451+0000","last_fullsized":"2024-09-22T03:26:03.759451+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"39'807","reported_seq":844,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:10.983771+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:10.983771+0000","last_peered":"2024-09-22T03:26:10.983771+0000","last_clean":"2024-09-22T03:26:10.983771+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:10.983771+0000","last_undegraded":"2024-09-22T03:26:10.983771+0000","last_fullsized":"2024-09-22T03:26:10.983771+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.747920+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:03.747920+0000","last_peered":"2024-09-22T03:26:03.747920+0000","last_clean":"2024-09-22T03:26:03.747920+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:03.747920+0000","last_undegraded":"2024-09-22T03:26:03.747920+0000","last_fullsized":"2024-09-22T03:26:03.747920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"39'919","reported_seq":956,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.516790+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:11.516790+0000","last_peered":"2024-09-22T03:26:11.516790+0000","last_clean":"2024-09-22T03:26:11.516790+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:11.516790+0000","last_undegraded":"2024-09-22T03:26:11.516790+0000","last_fullsized":"2024-09-22T03:26:11.516790+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":920,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.757769+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:03.757769+0000","last_peered":"2024-09-22T03:26:03.757769+0000","last_clean":"2024-09-22T03:26:03.757769+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:03.757769+0000","last_undegraded":"2024-09-22T03:26:03.757769+0000","last_fullsized":"2024-09-22T03:26:03.757769+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"39'703","reported_seq":740,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:11.784792+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:11.784792+0000","last_peered":"2024-09-22T03:26:11.784792+0000","last_clean":"2024-09-22T03:26:11.784792+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:11.784792+0000","last_undegraded":"2024-09-22T03:26:11.784792+0000","last_fullsized":"2024-09-22T03:26:11.784792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-22T03:26:03.772930+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:03.772930+0000","last_peered":"2024-09-22T03:26:03.772930+0000","last_clean":"2024-09-22T03:26:03.772930+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:03.772930+0000","last_undegraded":"2024-09-22T03:26:03.772930+0000","last_fullsized":"2024-09-22T03:26:03.772930+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":103030784,"num_objects":1573,"num_object_clones":0,"num_object_copies":4719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12577,"num_write_kb":100616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213563338752,"internally_reserved":0,"allocated":648155136,"data_stored":648155136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12561,"ondisk_log_size":12561,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213563338752,"internally_reserved":0,"allocated":648155136,"data_stored":648155136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53378068480,"internally_reserved":0,"allocated":174804992,"data_stored":174804992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":170548,"kb_used_data":170548,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52127180,"statfs":{"total":53552873472,"available":53378232320,"internally_reserved":0,"allocated":174641152,"data_stored":174641152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":131068,"kb_used_data":131068,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166660,"statfs":{"total":53552873472,"available":53418659840,"internally_reserved":0,"allocated":134213632,"data_stored":134213632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":168056,"kb_used_data":168056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52129672,"statfs":{"total":53552873472,"available":53380784128,"internally_reserved":0,"allocated":172089344,"data_stored":172089344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":162772,"kb_used_data":162772,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52134956,"statfs":{"total":53552873472,"available":53386194944,"internally_reserved":0,"allocated":166678528,"data_stored":166678528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53378068480,"internally_reserved":0,"allocated":174804992,"data_stored":174804992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53386170368,"internally_reserved":0,"allocated":166703104,"data_stored":166703104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53380612096,"internally_reserved":0,"allocated":172261376,"data_stored":172261376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53418487808,"internally_reserved":0,"allocated":134385664,"data_stored":134385664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53378068480,"internally_reserved":0,"allocated":174804992,"data_stored":174804992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53386170368,"internally_reserved":0,"allocated":166703104,"data_stored":166703104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53380612096,"internally_reserved":0,"allocated":172261376,"data_stored":172261376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53418487808,"internally_reserved":0,"allocated":134385664,"data_stored":134385664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53378068480,"internally_reserved":0,"allocated":174804992,"data_stored":174804992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:17.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 13316 13300 1.42325 0.953125 0.126618 0.0877537 2024-09-22T03:26:18.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 13464 13448 1.41964 1.15625 0.215351 0.0879063 2024-09-22T03:26:18.517 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:18.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:18.629+0000 7fac4ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:18.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:18.634+0000 7fac4ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:19.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 13628 13612 1.41779 1.28125 0.135656 0.0880747 2024-09-22T03:26:20.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 13816 13800 1.41846 1.46875 0.0607945 0.0880585 2024-09-22T03:26:20.810 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:20.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:20.921+0000 7f6bb070e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:20.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:20.927+0000 7f6bb070e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:21.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 13959 13943 1.41455 1.11719 0.0416989 0.0882869 2024-09-22T03:26:21.585 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:21.585 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:21.585 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:21.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:21.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:21.697+0000 7f3e71ac3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:21.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:21.712+0000 7f3e71ac3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:21.762 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:21.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:21.766 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:21.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:21.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:21.880+0000 7efc30997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:21.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:21.890+0000 7efc30997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:21.965 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:21.965 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:21.965 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:21.965 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:21.965 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:21.965 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:21.966 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:21.966 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:21.966 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:21.966 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:21.966 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:21.966 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:21.968 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:21.968 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:21.969 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:22.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:22.080+0000 7ff1b7f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:22.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:22.091+0000 7ff1b7f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:22.116 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 16 14111 14095 1.41164 1.1875 0.077682 0.0884837 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:22.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:22.172 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:22.172 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:22.172 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:22.175 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:22.176 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:22.176 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:22.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:22.333+0000 7f3e96917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:22.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:22.348+0000 7f3e96917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:22.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:22.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:22.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:22.375 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:22.378 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:22.378 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:23.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 14276 14260 1.41008 1.28906 0.106203 0.088594 2024-09-22T03:26:23.162 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:23.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:23.314+0000 7f9617ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:23.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:23.321+0000 7f9617ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:24.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:26:24.116661+0000 min lat: 0.0119124 max lat: 0.485942 avg lat: 0.0887714 2024-09-22T03:26:24.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:26:24.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 14416 14400 1.40613 1.09375 0.287859 0.0887714 2024-09-22T03:26:25.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 14582 14566 1.40478 1.29688 0.0524791 0.0889275 2024-09-22T03:26:25.695 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:25.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:25.857+0000 7f36a626c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:25.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:25.863+0000 7f36a626c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:26.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 14764 14748 1.40498 1.42188 0.0931925 0.0889041 2024-09-22T03:26:27.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 14920 14904 1.40274 1.21875 0.0537398 0.0890444 2024-09-22T03:26:27.380 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:27.380 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:26:27.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:27.490+0000 7f50fca4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:27.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:27.501+0000 7f50fca4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:28.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:28.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 15067 15051 1.39971 1.14844 0.200849 0.0891776 2024-09-22T03:26:28.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:28.191+0000 7f1960f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:28.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:28.197+0000 7f1960f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:28.704 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:26:28.720 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:26:28.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:28.832+0000 7f1b39c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:28.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:28.841+0000 7f1b39c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:29.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 15199 15183 1.39537 1.03125 0.0815379 0.0895265 2024-09-22T03:26:29.708 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:26:29.726 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:26:29.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:29.849+0000 7f0105197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:29.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:29.862+0000 7f0105197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:30.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 15332 15316 1.39123 1.03906 0.0497849 0.0897775 2024-09-22T03:26:30.381 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:30.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:30.516+0000 7fd27ad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:30.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:30.521+0000 7fd27ad0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:30.712 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:26:30.737 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:26:30.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:30.878+0000 7f246f6ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:30.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:30.891+0000 7f246f6ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:31.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 15489 15473 1.38934 1.22656 0.0846283 0.0899019 2024-09-22T03:26:31.716 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:26:31.737 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:26:31.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:31.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:31.862+0000 7f443e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:31.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:31.870+0000 7f443e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:32.117 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 15646 15630 1.38749 1.22656 0.0725044 0.0900478 2024-09-22T03:26:32.148 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:32.149 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:32.164 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-22T03:26:31.973346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128459315,"num_objects":1956,"num_object_clones":0,"num_object_copies":5868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":15748,"num_write_kb":126768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15705,"ondisk_log_size":15705,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":784500,"kb_used_data":784500,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208406412,"statfs":{"total":214211493888,"available":213408165888,"internally_reserved":0,"allocated":803328000,"data_stored":803328000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5914624,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":722,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000684"},"pg_stats":[{"pgid":"4.f","version":"42'1144","reported_seq":1192,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.761340+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:31.761340+0000","last_peered":"2024-09-22T03:26:31.761340+0000","last_clean":"2024-09-22T03:26:31.761340+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:31.761340+0000","last_undegraded":"2024-09-22T03:26:31.761340+0000","last_fullsized":"2024-09-22T03:26:31.761340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1144,"log_dups_size":0,"ondisk_log_size":1144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9371648,"num_objects":143,"num_object_clones":0,"num_object_copies":429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1144,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"41'848","reported_seq":896,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.763565+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:31.763565+0000","last_peered":"2024-09-22T03:26:31.763565+0000","last_clean":"2024-09-22T03:26:31.763565+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:31.763565+0000","last_undegraded":"2024-09-22T03:26:31.763565+0000","last_fullsized":"2024-09-22T03:26:31.763565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"42'951","reported_seq":1000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.751681+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:31.751681+0000","last_peered":"2024-09-22T03:26:31.751681+0000","last_clean":"2024-09-22T03:26:31.751681+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:31.751681+0000","last_undegraded":"2024-09-22T03:26:31.751681+0000","last_fullsized":"2024-09-22T03:26:31.751681+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"42'920","reported_seq":968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.733055+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:31.733055+0000","last_peered":"2024-09-22T03:26:31.733055+0000","last_clean":"2024-09-22T03:26:31.733055+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:31.733055+0000","last_undegraded":"2024-09-22T03:26:31.733055+0000","last_fullsized":"2024-09-22T03:26:31.733055+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":920,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"42'944","reported_seq":993,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.756457+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:31.756457+0000","last_peered":"2024-09-22T03:26:31.756457+0000","last_clean":"2024-09-22T03:26:31.756457+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:31.756457+0000","last_undegraded":"2024-09-22T03:26:31.756457+0000","last_fullsized":"2024-09-22T03:26:31.756457+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":7560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"42'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.744375+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:31.744375+0000","last_peered":"2024-09-22T03:26:31.744375+0000","last_clean":"2024-09-22T03:26:31.744375+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:31.744375+0000","last_undegraded":"2024-09-22T03:26:31.744375+0000","last_fullsized":"2024-09-22T03:26:31.744375+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"42'936","reported_seq":984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.747692+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:31.747692+0000","last_peered":"2024-09-22T03:26:31.747692+0000","last_clean":"2024-09-22T03:26:31.747692+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:31.747692+0000","last_undegraded":"2024-09-22T03:26:31.747692+0000","last_fullsized":"2024-09-22T03:26:31.747692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"41'968","reported_seq":1016,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745553+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:31.745553+0000","last_peered":"2024-09-22T03:26:31.745553+0000","last_clean":"2024-09-22T03:26:31.745553+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:31.745553+0000","last_undegraded":"2024-09-22T03:26:31.745553+0000","last_fullsized":"2024-09-22T03:26:31.745553+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"41'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.741207+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:31.741207+0000","last_peered":"2024-09-22T03:26:31.741207+0000","last_clean":"2024-09-22T03:26:31.741207+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:31.741207+0000","last_undegraded":"2024-09-22T03:26:31.741207+0000","last_fullsized":"2024-09-22T03:26:31.741207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":232,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.730377+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:31.730377+0000","last_peered":"2024-09-22T03:26:31.730377+0000","last_clean":"2024-09-22T03:26:31.730377+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:31.730377+0000","last_undegraded":"2024-09-22T03:26:31.730377+0000","last_fullsized":"2024-09-22T03:26:31.730377+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.743975+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:31.743975+0000","last_peered":"2024-09-22T03:26:31.743975+0000","last_clean":"2024-09-22T03:26:31.743975+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:31.743975+0000","last_undegraded":"2024-09-22T03:26:31.743975+0000","last_fullsized":"2024-09-22T03:26:31.743975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"43'991","reported_seq":1040,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.888116+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:31.888116+0000","last_peered":"2024-09-22T03:26:31.888116+0000","last_clean":"2024-09-22T03:26:31.888116+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:31.888116+0000","last_undegraded":"2024-09-22T03:26:31.888116+0000","last_fullsized":"2024-09-22T03:26:31.888116+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.754948+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:31.754948+0000","last_peered":"2024-09-22T03:26:31.754948+0000","last_clean":"2024-09-22T03:26:31.754948+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:31.754948+0000","last_undegraded":"2024-09-22T03:26:31.754948+0000","last_fullsized":"2024-09-22T03:26:31.754948+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"42'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762003+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:31.762003+0000","last_peered":"2024-09-22T03:26:31.762003+0000","last_clean":"2024-09-22T03:26:31.762003+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:31.762003+0000","last_undegraded":"2024-09-22T03:26:31.762003+0000","last_fullsized":"2024-09-22T03:26:31.762003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.741689+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:31.741689+0000","last_peered":"2024-09-22T03:26:31.741689+0000","last_clean":"2024-09-22T03:26:31.741689+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:31.741689+0000","last_undegraded":"2024-09-22T03:26:31.741689+0000","last_fullsized":"2024-09-22T03:26:31.741689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"42'928","reported_seq":976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.760701+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:31.760701+0000","last_peered":"2024-09-22T03:26:31.760701+0000","last_clean":"2024-09-22T03:26:31.760701+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:31.760701+0000","last_undegraded":"2024-09-22T03:26:31.760701+0000","last_fullsized":"2024-09-22T03:26:31.760701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.750823+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:31.750823+0000","last_peered":"2024-09-22T03:26:31.750823+0000","last_clean":"2024-09-22T03:26:31.750823+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:31.750823+0000","last_undegraded":"2024-09-22T03:26:31.750823+0000","last_fullsized":"2024-09-22T03:26:31.750823+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"41'952","reported_seq":1000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745721+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:31.745721+0000","last_peered":"2024-09-22T03:26:31.745721+0000","last_clean":"2024-09-22T03:26:31.745721+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:31.745721+0000","last_undegraded":"2024-09-22T03:26:31.745721+0000","last_fullsized":"2024-09-22T03:26:31.745721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762984+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:31.762984+0000","last_peered":"2024-09-22T03:26:31.762984+0000","last_clean":"2024-09-22T03:26:31.762984+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:31.762984+0000","last_undegraded":"2024-09-22T03:26:31.762984+0000","last_fullsized":"2024-09-22T03:26:31.762984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"43'1031","reported_seq":1080,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.801361+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:31.801361+0000","last_peered":"2024-09-22T03:26:31.801361+0000","last_clean":"2024-09-22T03:26:31.801361+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:31.801361+0000","last_undegraded":"2024-09-22T03:26:31.801361+0000","last_fullsized":"2024-09-22T03:26:31.801361+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1032,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.733039+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:31.733039+0000","last_peered":"2024-09-22T03:26:31.733039+0000","last_clean":"2024-09-22T03:26:31.733039+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:31.733039+0000","last_undegraded":"2024-09-22T03:26:31.733039+0000","last_fullsized":"2024-09-22T03:26:31.733039+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"41'1056","reported_seq":1104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.752150+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:31.752150+0000","last_peered":"2024-09-22T03:26:31.752150+0000","last_clean":"2024-09-22T03:26:31.752150+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:31.752150+0000","last_undegraded":"2024-09-22T03:26:31.752150+0000","last_fullsized":"2024-09-22T03:26:31.752150+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8650752,"num_objects":132,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1056,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762731+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:31.762731+0000","last_peered":"2024-09-22T03:26:31.762731+0000","last_clean":"2024-09-22T03:26:31.762731+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:31.762731+0000","last_undegraded":"2024-09-22T03:26:31.762731+0000","last_fullsized":"2024-09-22T03:26:31.762731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"42'888","reported_seq":936,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.747252+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:31.747252+0000","last_peered":"2024-09-22T03:26:31.747252+0000","last_clean":"2024-09-22T03:26:31.747252+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:31.747252+0000","last_undegraded":"2024-09-22T03:26:31.747252+0000","last_fullsized":"2024-09-22T03:26:31.747252+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7274496,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745006+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:31.745006+0000","last_peered":"2024-09-22T03:26:31.745006+0000","last_clean":"2024-09-22T03:26:31.745006+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:31.745006+0000","last_undegraded":"2024-09-22T03:26:31.745006+0000","last_fullsized":"2024-09-22T03:26:31.745006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":127868928,"num_objects":1952,"num_object_clones":0,"num_object_copies":5856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":124872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213407969280,"internally_reserved":0,"allocated":803524608,"data_stored":803524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15605,"ondisk_log_size":15605,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213407969280,"internally_reserved":0,"allocated":803524608,"data_stored":803524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":212372,"kb_used_data":212372,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52085356,"statfs":{"total":53552873472,"available":53335404544,"internally_reserved":0,"allocated":217468928,"data_stored":217468928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":165216,"kb_used_data":165216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52132512,"statfs":{"total":53552873472,"available":53383692288,"internally_reserved":0,"allocated":169181184,"data_stored":169181184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":207920,"kb_used_data":207920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52089808,"statfs":{"total":53552873472,"available":53339963392,"internally_reserved":0,"allocated":212910080,"data_stored":212910080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":198992,"kb_used_data":198992,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52098736,"statfs":{"total":53552873472,"available":53349105664,"internally_reserved":0,"allocated":203767808,"data_stored":203767808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53349097472,"internally_reserved":0,"allocated":203776000,"data_stored":203776000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53339881472,"internally_reserved":0,"allocated":212992000,"data_stored":212992000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53383610368,"internally_reserved":0,"allocated":169263104,"data_stored":169263104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53349097472,"internally_reserved":0,"allocated":203776000,"data_stored":203776000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53339881472,"internally_reserved":0,"allocated":212992000,"data_stored":212992000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53383610368,"internally_reserved":0,"allocated":169263104,"data_stored":169263104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:32.166 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:32.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:32.281+0000 7f5223117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:32.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:32.290+0000 7f5223117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:32.587 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:32.587 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:32.608 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-22T03:26:31.973346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128459315,"num_objects":1956,"num_object_clones":0,"num_object_copies":5868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":15748,"num_write_kb":126768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15705,"ondisk_log_size":15705,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":784500,"kb_used_data":784500,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208406412,"statfs":{"total":214211493888,"available":213408165888,"internally_reserved":0,"allocated":803328000,"data_stored":803328000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5914624,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":722,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000684"},"pg_stats":[{"pgid":"4.f","version":"42'1144","reported_seq":1192,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.761340+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:31.761340+0000","last_peered":"2024-09-22T03:26:31.761340+0000","last_clean":"2024-09-22T03:26:31.761340+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:31.761340+0000","last_undegraded":"2024-09-22T03:26:31.761340+0000","last_fullsized":"2024-09-22T03:26:31.761340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1144,"log_dups_size":0,"ondisk_log_size":1144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9371648,"num_objects":143,"num_object_clones":0,"num_object_copies":429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1144,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"41'848","reported_seq":896,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.763565+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:31.763565+0000","last_peered":"2024-09-22T03:26:31.763565+0000","last_clean":"2024-09-22T03:26:31.763565+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:31.763565+0000","last_undegraded":"2024-09-22T03:26:31.763565+0000","last_fullsized":"2024-09-22T03:26:31.763565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"42'951","reported_seq":1000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.751681+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:31.751681+0000","last_peered":"2024-09-22T03:26:31.751681+0000","last_clean":"2024-09-22T03:26:31.751681+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:31.751681+0000","last_undegraded":"2024-09-22T03:26:31.751681+0000","last_fullsized":"2024-09-22T03:26:31.751681+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"42'920","reported_seq":968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.733055+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:31.733055+0000","last_peered":"2024-09-22T03:26:31.733055+0000","last_clean":"2024-09-22T03:26:31.733055+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:31.733055+0000","last_undegraded":"2024-09-22T03:26:31.733055+0000","last_fullsized":"2024-09-22T03:26:31.733055+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":920,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"42'944","reported_seq":993,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.756457+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:31.756457+0000","last_peered":"2024-09-22T03:26:31.756457+0000","last_clean":"2024-09-22T03:26:31.756457+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:31.756457+0000","last_undegraded":"2024-09-22T03:26:31.756457+0000","last_fullsized":"2024-09-22T03:26:31.756457+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":7560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"42'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.744375+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:31.744375+0000","last_peered":"2024-09-22T03:26:31.744375+0000","last_clean":"2024-09-22T03:26:31.744375+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:31.744375+0000","last_undegraded":"2024-09-22T03:26:31.744375+0000","last_fullsized":"2024-09-22T03:26:31.744375+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"42'936","reported_seq":984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.747692+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:31.747692+0000","last_peered":"2024-09-22T03:26:31.747692+0000","last_clean":"2024-09-22T03:26:31.747692+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:31.747692+0000","last_undegraded":"2024-09-22T03:26:31.747692+0000","last_fullsized":"2024-09-22T03:26:31.747692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"41'968","reported_seq":1016,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745553+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:31.745553+0000","last_peered":"2024-09-22T03:26:31.745553+0000","last_clean":"2024-09-22T03:26:31.745553+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:31.745553+0000","last_undegraded":"2024-09-22T03:26:31.745553+0000","last_fullsized":"2024-09-22T03:26:31.745553+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"41'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.741207+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:31.741207+0000","last_peered":"2024-09-22T03:26:31.741207+0000","last_clean":"2024-09-22T03:26:31.741207+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:31.741207+0000","last_undegraded":"2024-09-22T03:26:31.741207+0000","last_fullsized":"2024-09-22T03:26:31.741207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":232,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.730377+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:31.730377+0000","last_peered":"2024-09-22T03:26:31.730377+0000","last_clean":"2024-09-22T03:26:31.730377+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:31.730377+0000","last_undegraded":"2024-09-22T03:26:31.730377+0000","last_fullsized":"2024-09-22T03:26:31.730377+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.743975+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:31.743975+0000","last_peered":"2024-09-22T03:26:31.743975+0000","last_clean":"2024-09-22T03:26:31.743975+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:31.743975+0000","last_undegraded":"2024-09-22T03:26:31.743975+0000","last_fullsized":"2024-09-22T03:26:31.743975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"43'991","reported_seq":1040,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.888116+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:31.888116+0000","last_peered":"2024-09-22T03:26:31.888116+0000","last_clean":"2024-09-22T03:26:31.888116+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:31.888116+0000","last_undegraded":"2024-09-22T03:26:31.888116+0000","last_fullsized":"2024-09-22T03:26:31.888116+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.754948+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:31.754948+0000","last_peered":"2024-09-22T03:26:31.754948+0000","last_clean":"2024-09-22T03:26:31.754948+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:31.754948+0000","last_undegraded":"2024-09-22T03:26:31.754948+0000","last_fullsized":"2024-09-22T03:26:31.754948+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"42'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762003+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:31.762003+0000","last_peered":"2024-09-22T03:26:31.762003+0000","last_clean":"2024-09-22T03:26:31.762003+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:31.762003+0000","last_undegraded":"2024-09-22T03:26:31.762003+0000","last_fullsized":"2024-09-22T03:26:31.762003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.741689+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:31.741689+0000","last_peered":"2024-09-22T03:26:31.741689+0000","last_clean":"2024-09-22T03:26:31.741689+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:31.741689+0000","last_undegraded":"2024-09-22T03:26:31.741689+0000","last_fullsized":"2024-09-22T03:26:31.741689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"42'928","reported_seq":976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.760701+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:31.760701+0000","last_peered":"2024-09-22T03:26:31.760701+0000","last_clean":"2024-09-22T03:26:31.760701+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:31.760701+0000","last_undegraded":"2024-09-22T03:26:31.760701+0000","last_fullsized":"2024-09-22T03:26:31.760701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.750823+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:31.750823+0000","last_peered":"2024-09-22T03:26:31.750823+0000","last_clean":"2024-09-22T03:26:31.750823+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:31.750823+0000","last_undegraded":"2024-09-22T03:26:31.750823+0000","last_fullsized":"2024-09-22T03:26:31.750823+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"41'952","reported_seq":1000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745721+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:31.745721+0000","last_peered":"2024-09-22T03:26:31.745721+0000","last_clean":"2024-09-22T03:26:31.745721+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:31.745721+0000","last_undegraded":"2024-09-22T03:26:31.745721+0000","last_fullsized":"2024-09-22T03:26:31.745721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762984+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:31.762984+0000","last_peered":"2024-09-22T03:26:31.762984+0000","last_clean":"2024-09-22T03:26:31.762984+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:31.762984+0000","last_undegraded":"2024-09-22T03:26:31.762984+0000","last_fullsized":"2024-09-22T03:26:31.762984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"43'1031","reported_seq":1080,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.801361+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:31.801361+0000","last_peered":"2024-09-22T03:26:31.801361+0000","last_clean":"2024-09-22T03:26:31.801361+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:31.801361+0000","last_undegraded":"2024-09-22T03:26:31.801361+0000","last_fullsized":"2024-09-22T03:26:31.801361+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1032,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.733039+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:31.733039+0000","last_peered":"2024-09-22T03:26:31.733039+0000","last_clean":"2024-09-22T03:26:31.733039+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:31.733039+0000","last_undegraded":"2024-09-22T03:26:31.733039+0000","last_fullsized":"2024-09-22T03:26:31.733039+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"41'1056","reported_seq":1104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.752150+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:31.752150+0000","last_peered":"2024-09-22T03:26:31.752150+0000","last_clean":"2024-09-22T03:26:31.752150+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:31.752150+0000","last_undegraded":"2024-09-22T03:26:31.752150+0000","last_fullsized":"2024-09-22T03:26:31.752150+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8650752,"num_objects":132,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1056,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762731+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:31.762731+0000","last_peered":"2024-09-22T03:26:31.762731+0000","last_clean":"2024-09-22T03:26:31.762731+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:31.762731+0000","last_undegraded":"2024-09-22T03:26:31.762731+0000","last_fullsized":"2024-09-22T03:26:31.762731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"42'888","reported_seq":936,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.747252+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:31.747252+0000","last_peered":"2024-09-22T03:26:31.747252+0000","last_clean":"2024-09-22T03:26:31.747252+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:31.747252+0000","last_undegraded":"2024-09-22T03:26:31.747252+0000","last_fullsized":"2024-09-22T03:26:31.747252+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7274496,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745006+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:31.745006+0000","last_peered":"2024-09-22T03:26:31.745006+0000","last_clean":"2024-09-22T03:26:31.745006+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:31.745006+0000","last_undegraded":"2024-09-22T03:26:31.745006+0000","last_fullsized":"2024-09-22T03:26:31.745006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":127868928,"num_objects":1952,"num_object_clones":0,"num_object_copies":5856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":124872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213407969280,"internally_reserved":0,"allocated":803524608,"data_stored":803524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15605,"ondisk_log_size":15605,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213407969280,"internally_reserved":0,"allocated":803524608,"data_stored":803524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":212372,"kb_used_data":212372,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52085356,"statfs":{"total":53552873472,"available":53335404544,"internally_reserved":0,"allocated":217468928,"data_stored":217468928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":165216,"kb_used_data":165216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52132512,"statfs":{"total":53552873472,"available":53383692288,"internally_reserved":0,"allocated":169181184,"data_stored":169181184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":207920,"kb_used_data":207920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52089808,"statfs":{"total":53552873472,"available":53339963392,"internally_reserved":0,"allocated":212910080,"data_stored":212910080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":198992,"kb_used_data":198992,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52098736,"statfs":{"total":53552873472,"available":53349105664,"internally_reserved":0,"allocated":203767808,"data_stored":203767808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53349097472,"internally_reserved":0,"allocated":203776000,"data_stored":203776000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53339881472,"internally_reserved":0,"allocated":212992000,"data_stored":212992000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53383610368,"internally_reserved":0,"allocated":169263104,"data_stored":169263104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53349097472,"internally_reserved":0,"allocated":203776000,"data_stored":203776000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53339881472,"internally_reserved":0,"allocated":212992000,"data_stored":212992000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53383610368,"internally_reserved":0,"allocated":169263104,"data_stored":169263104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:32.609 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:26:32.609 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:32.609 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:32.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:32.735+0000 7fc96342c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:32.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:32.744+0000 7fc96342c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:32.834 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:32.946+0000 7fa29eeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:32.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:32.951+0000 7fa29eeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:32.955 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:32.956 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:32.970 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-22T03:26:31.973346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128459315,"num_objects":1956,"num_object_clones":0,"num_object_copies":5868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":15748,"num_write_kb":126768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15705,"ondisk_log_size":15705,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":784500,"kb_used_data":784500,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208406412,"statfs":{"total":214211493888,"available":213408165888,"internally_reserved":0,"allocated":803328000,"data_stored":803328000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5914624,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":722,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000684"},"pg_stats":[{"pgid":"4.f","version":"42'1144","reported_seq":1192,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.761340+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:31.761340+0000","last_peered":"2024-09-22T03:26:31.761340+0000","last_clean":"2024-09-22T03:26:31.761340+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:31.761340+0000","last_undegraded":"2024-09-22T03:26:31.761340+0000","last_fullsized":"2024-09-22T03:26:31.761340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1144,"log_dups_size":0,"ondisk_log_size":1144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9371648,"num_objects":143,"num_object_clones":0,"num_object_copies":429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1144,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"41'848","reported_seq":896,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.763565+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:31.763565+0000","last_peered":"2024-09-22T03:26:31.763565+0000","last_clean":"2024-09-22T03:26:31.763565+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:31.763565+0000","last_undegraded":"2024-09-22T03:26:31.763565+0000","last_fullsized":"2024-09-22T03:26:31.763565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"42'951","reported_seq":1000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.751681+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:31.751681+0000","last_peered":"2024-09-22T03:26:31.751681+0000","last_clean":"2024-09-22T03:26:31.751681+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:31.751681+0000","last_undegraded":"2024-09-22T03:26:31.751681+0000","last_fullsized":"2024-09-22T03:26:31.751681+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"42'920","reported_seq":968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.733055+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:31.733055+0000","last_peered":"2024-09-22T03:26:31.733055+0000","last_clean":"2024-09-22T03:26:31.733055+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:31.733055+0000","last_undegraded":"2024-09-22T03:26:31.733055+0000","last_fullsized":"2024-09-22T03:26:31.733055+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":920,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"42'944","reported_seq":993,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.756457+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:31.756457+0000","last_peered":"2024-09-22T03:26:31.756457+0000","last_clean":"2024-09-22T03:26:31.756457+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:31.756457+0000","last_undegraded":"2024-09-22T03:26:31.756457+0000","last_fullsized":"2024-09-22T03:26:31.756457+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":7560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"42'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.744375+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:31.744375+0000","last_peered":"2024-09-22T03:26:31.744375+0000","last_clean":"2024-09-22T03:26:31.744375+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:31.744375+0000","last_undegraded":"2024-09-22T03:26:31.744375+0000","last_fullsized":"2024-09-22T03:26:31.744375+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"42'936","reported_seq":984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.747692+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:31.747692+0000","last_peered":"2024-09-22T03:26:31.747692+0000","last_clean":"2024-09-22T03:26:31.747692+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:31.747692+0000","last_undegraded":"2024-09-22T03:26:31.747692+0000","last_fullsized":"2024-09-22T03:26:31.747692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"41'968","reported_seq":1016,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745553+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:31.745553+0000","last_peered":"2024-09-22T03:26:31.745553+0000","last_clean":"2024-09-22T03:26:31.745553+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:31.745553+0000","last_undegraded":"2024-09-22T03:26:31.745553+0000","last_fullsized":"2024-09-22T03:26:31.745553+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"41'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.741207+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:31.741207+0000","last_peered":"2024-09-22T03:26:31.741207+0000","last_clean":"2024-09-22T03:26:31.741207+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:31.741207+0000","last_undegraded":"2024-09-22T03:26:31.741207+0000","last_fullsized":"2024-09-22T03:26:31.741207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":232,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.730377+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:31.730377+0000","last_peered":"2024-09-22T03:26:31.730377+0000","last_clean":"2024-09-22T03:26:31.730377+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:31.730377+0000","last_undegraded":"2024-09-22T03:26:31.730377+0000","last_fullsized":"2024-09-22T03:26:31.730377+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.743975+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:31.743975+0000","last_peered":"2024-09-22T03:26:31.743975+0000","last_clean":"2024-09-22T03:26:31.743975+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:31.743975+0000","last_undegraded":"2024-09-22T03:26:31.743975+0000","last_fullsized":"2024-09-22T03:26:31.743975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"43'991","reported_seq":1040,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.888116+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:31.888116+0000","last_peered":"2024-09-22T03:26:31.888116+0000","last_clean":"2024-09-22T03:26:31.888116+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:31.888116+0000","last_undegraded":"2024-09-22T03:26:31.888116+0000","last_fullsized":"2024-09-22T03:26:31.888116+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":124,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":7936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.754948+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:31.754948+0000","last_peered":"2024-09-22T03:26:31.754948+0000","last_clean":"2024-09-22T03:26:31.754948+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:31.754948+0000","last_undegraded":"2024-09-22T03:26:31.754948+0000","last_fullsized":"2024-09-22T03:26:31.754948+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"42'1016","reported_seq":1064,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762003+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:31.762003+0000","last_peered":"2024-09-22T03:26:31.762003+0000","last_clean":"2024-09-22T03:26:31.762003+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:31.762003+0000","last_undegraded":"2024-09-22T03:26:31.762003+0000","last_fullsized":"2024-09-22T03:26:31.762003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.741689+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:31.741689+0000","last_peered":"2024-09-22T03:26:31.741689+0000","last_clean":"2024-09-22T03:26:31.741689+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:31.741689+0000","last_undegraded":"2024-09-22T03:26:31.741689+0000","last_fullsized":"2024-09-22T03:26:31.741689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"42'928","reported_seq":976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.760701+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:31.760701+0000","last_peered":"2024-09-22T03:26:31.760701+0000","last_clean":"2024-09-22T03:26:31.760701+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:31.760701+0000","last_undegraded":"2024-09-22T03:26:31.760701+0000","last_fullsized":"2024-09-22T03:26:31.760701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.750823+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:31.750823+0000","last_peered":"2024-09-22T03:26:31.750823+0000","last_clean":"2024-09-22T03:26:31.750823+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:31.750823+0000","last_undegraded":"2024-09-22T03:26:31.750823+0000","last_fullsized":"2024-09-22T03:26:31.750823+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"41'952","reported_seq":1000,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745721+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:31.745721+0000","last_peered":"2024-09-22T03:26:31.745721+0000","last_clean":"2024-09-22T03:26:31.745721+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:31.745721+0000","last_undegraded":"2024-09-22T03:26:31.745721+0000","last_fullsized":"2024-09-22T03:26:31.745721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762984+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:31.762984+0000","last_peered":"2024-09-22T03:26:31.762984+0000","last_clean":"2024-09-22T03:26:31.762984+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:31.762984+0000","last_undegraded":"2024-09-22T03:26:31.762984+0000","last_fullsized":"2024-09-22T03:26:31.762984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"43'1031","reported_seq":1080,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.801361+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:31.801361+0000","last_peered":"2024-09-22T03:26:31.801361+0000","last_clean":"2024-09-22T03:26:31.801361+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:31.801361+0000","last_undegraded":"2024-09-22T03:26:31.801361+0000","last_fullsized":"2024-09-22T03:26:31.801361+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1032,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.733039+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:31.733039+0000","last_peered":"2024-09-22T03:26:31.733039+0000","last_clean":"2024-09-22T03:26:31.733039+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:31.733039+0000","last_undegraded":"2024-09-22T03:26:31.733039+0000","last_fullsized":"2024-09-22T03:26:31.733039+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"41'1056","reported_seq":1104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.752150+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:31.752150+0000","last_peered":"2024-09-22T03:26:31.752150+0000","last_clean":"2024-09-22T03:26:31.752150+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:31.752150+0000","last_undegraded":"2024-09-22T03:26:31.752150+0000","last_fullsized":"2024-09-22T03:26:31.752150+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1056,"log_dups_size":0,"ondisk_log_size":1056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8650752,"num_objects":132,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1056,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.762731+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:31.762731+0000","last_peered":"2024-09-22T03:26:31.762731+0000","last_clean":"2024-09-22T03:26:31.762731+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:31.762731+0000","last_undegraded":"2024-09-22T03:26:31.762731+0000","last_fullsized":"2024-09-22T03:26:31.762731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"42'888","reported_seq":936,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.747252+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:31.747252+0000","last_peered":"2024-09-22T03:26:31.747252+0000","last_clean":"2024-09-22T03:26:31.747252+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:31.747252+0000","last_undegraded":"2024-09-22T03:26:31.747252+0000","last_fullsized":"2024-09-22T03:26:31.747252+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7274496,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-22T03:26:31.745006+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:31.745006+0000","last_peered":"2024-09-22T03:26:31.745006+0000","last_clean":"2024-09-22T03:26:31.745006+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:31.745006+0000","last_undegraded":"2024-09-22T03:26:31.745006+0000","last_fullsized":"2024-09-22T03:26:31.745006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":127868928,"num_objects":1952,"num_object_clones":0,"num_object_copies":5856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":124872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213407969280,"internally_reserved":0,"allocated":803524608,"data_stored":803524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15605,"ondisk_log_size":15605,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213407969280,"internally_reserved":0,"allocated":803524608,"data_stored":803524608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":212372,"kb_used_data":212372,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52085356,"statfs":{"total":53552873472,"available":53335404544,"internally_reserved":0,"allocated":217468928,"data_stored":217468928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":165216,"kb_used_data":165216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52132512,"statfs":{"total":53552873472,"available":53383692288,"internally_reserved":0,"allocated":169181184,"data_stored":169181184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":207920,"kb_used_data":207920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52089808,"statfs":{"total":53552873472,"available":53339963392,"internally_reserved":0,"allocated":212910080,"data_stored":212910080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":198992,"kb_used_data":198992,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52098736,"statfs":{"total":53552873472,"available":53349105664,"internally_reserved":0,"allocated":203767808,"data_stored":203767808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53349097472,"internally_reserved":0,"allocated":203776000,"data_stored":203776000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53339881472,"internally_reserved":0,"allocated":212992000,"data_stored":212992000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53383610368,"internally_reserved":0,"allocated":169263104,"data_stored":169263104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53349097472,"internally_reserved":0,"allocated":203776000,"data_stored":203776000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53339881472,"internally_reserved":0,"allocated":212992000,"data_stored":212992000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53383610368,"internally_reserved":0,"allocated":169263104,"data_stored":169263104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53335379968,"internally_reserved":0,"allocated":217493504,"data_stored":217493504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:33.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 15782 15766 1.38383 1.0625 0.17556 0.0902689 2024-09-22T03:26:34.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 15 15932 15917 1.38156 1.17969 0.213078 0.0903427 2024-09-22T03:26:35.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.1584 2024-09-22T03:26:35.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 15932 2024-09-22T03:26:35.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:26:35.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:26:35.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.38056 2024-09-22T03:26:35.118 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.183311 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.75 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 0.953125 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 176 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 23.4638 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 224 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 122 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.0904571 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.0506547 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.485942 2024-09-22T03:26:35.119 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0119124 2024-09-22T03:26:35.140 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:26:35.172 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:35.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:35.328+0000 7f0359397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:35.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:35.335+0000 7f0359397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:37.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:37.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:37.824+0000 7f27528d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:37.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:37.831+0000 7f27528d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:37.972 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:37.972 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:37.972 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:37.972 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:38.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.101+0000 7ff330618640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.111+0000 7ff330618640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.147 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:38.147 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:38.147 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:38.148 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:38.152 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:38.152 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:38.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:38.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.332+0000 7efc0d018640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.343+0000 7efc0d018640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:38.413 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:38.416 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:38.416 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:38.417 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:38.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.569+0000 7febfef0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.581+0000 7febfef0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.608 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:38.609 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:38.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:38.610 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:38.612 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:38.613 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:38.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:38.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.772+0000 7fb625397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:38.783+0000 7fb625397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:38.814 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:38.814 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:38.814 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:38.815 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:38.818 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:38.819 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:40.122 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:40.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:40.272+0000 7fdf7a22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:40.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:40.281+0000 7fdf7a22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:42.489 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:42.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:42.613+0000 7efd94b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:42.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:42.619+0000 7efd94b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:43.820 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:43.820 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:26:43.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:43.960+0000 7fec35117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:43.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:43.971+0000 7fec35117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:44.740 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:26:44.765 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:26:44.879 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:44.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:44.897+0000 7f7618197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:44.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:44.913+0000 7f7618197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:45.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:45.009+0000 7fe69b22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:45.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:45.016+0000 7fe69b22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:45.745 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:26:45.768 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:26:45.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:45.895+0000 7f9032117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:45.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:45.906+0000 7f9032117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:46.584 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 1992 objects 2024-09-22T03:26:46.584 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :11.4443 2024-09-22T03:26:46.752 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:26:46.769 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:26:46.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:46.868+0000 7fe394ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:46.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:46.877+0000 7fe394ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:47.211 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:47.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:47.310+0000 7fbe83a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:47.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:47.314+0000 7fbe83a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:47.752 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:26:47.774 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:26:47.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:47.776 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-22T03:26:47.776 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-22T03:26:47.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:47.875+0000 7ff25cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:47.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:47.882+0000 7ff25cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:47.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:47.921+0000 7fb120263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:47.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:47.932+0000 7fb120263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.068 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:48.069 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:48.081 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-22T03:26:47.976363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":18065,"num_write_kb":129353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18026,"ondisk_log_size":18026,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":33708,"kb_used_data":33708,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209157204,"statfs":{"total":214211493888,"available":214176976896,"internally_reserved":0,"allocated":34516992,"data_stored":34516992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47349784,"num_objects":-724,"num_object_clones":0,"num_object_copies":-2172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":724,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000678"},"pg_stats":[{"pgid":"4.f","version":"45'1305","reported_seq":1362,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.762834+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:46.762834+0000","last_peered":"2024-09-22T03:26:46.762834+0000","last_clean":"2024-09-22T03:26:46.762834+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:46.762834+0000","last_undegraded":"2024-09-22T03:26:46.762834+0000","last_fullsized":"2024-09-22T03:26:46.762834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1305,"log_dups_size":0,"ondisk_log_size":1305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1305,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"45'972","reported_seq":1029,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.767089+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:46.767089+0000","last_peered":"2024-09-22T03:26:46.767089+0000","last_clean":"2024-09-22T03:26:46.767089+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:46.767089+0000","last_undegraded":"2024-09-22T03:26:46.767089+0000","last_fullsized":"2024-09-22T03:26:46.767089+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":972,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"45'1100","reported_seq":1158,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.771258+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:46.771258+0000","last_peered":"2024-09-22T03:26:46.771258+0000","last_clean":"2024-09-22T03:26:46.771258+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:46.771258+0000","last_undegraded":"2024-09-22T03:26:46.771258+0000","last_fullsized":"2024-09-22T03:26:46.771258+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1100,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"45'1053","reported_seq":1110,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.764638+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:46.764638+0000","last_peered":"2024-09-22T03:26:46.764638+0000","last_clean":"2024-09-22T03:26:46.764638+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:46.764638+0000","last_undegraded":"2024-09-22T03:26:46.764638+0000","last_fullsized":"2024-09-22T03:26:46.764638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1053,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"45'1071","reported_seq":1128,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769783+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:46.769783+0000","last_peered":"2024-09-22T03:26:46.769783+0000","last_clean":"2024-09-22T03:26:46.769783+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:46.769783+0000","last_undegraded":"2024-09-22T03:26:46.769783+0000","last_fullsized":"2024-09-22T03:26:46.769783+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1071,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"45'1161","reported_seq":1218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768388+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:46.768388+0000","last_peered":"2024-09-22T03:26:46.768388+0000","last_clean":"2024-09-22T03:26:46.768388+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:46.768388+0000","last_undegraded":"2024-09-22T03:26:46.768388+0000","last_fullsized":"2024-09-22T03:26:46.768388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1161,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"45'1053","reported_seq":1110,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.771800+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:46.771800+0000","last_peered":"2024-09-22T03:26:46.771800+0000","last_clean":"2024-09-22T03:26:46.771800+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:46.771800+0000","last_undegraded":"2024-09-22T03:26:46.771800+0000","last_fullsized":"2024-09-22T03:26:46.771800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1053,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"45'1175","reported_seq":1232,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.767220+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:46.767220+0000","last_peered":"2024-09-22T03:26:46.767220+0000","last_clean":"2024-09-22T03:26:46.767220+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:46.767220+0000","last_undegraded":"2024-09-22T03:26:46.767220+0000","last_fullsized":"2024-09-22T03:26:46.767220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1175,"log_dups_size":0,"ondisk_log_size":1175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1175,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"45'1143","reported_seq":1200,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.765308+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:46.765308+0000","last_peered":"2024-09-22T03:26:46.765308+0000","last_clean":"2024-09-22T03:26:46.765308+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:46.765308+0000","last_undegraded":"2024-09-22T03:26:46.765308+0000","last_fullsized":"2024-09-22T03:26:46.765308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1143,"log_dups_size":0,"ondisk_log_size":1143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1143,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":241,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.763012+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:46.763012+0000","last_peered":"2024-09-22T03:26:46.763012+0000","last_clean":"2024-09-22T03:26:46.763012+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:46.763012+0000","last_undegraded":"2024-09-22T03:26:46.763012+0000","last_fullsized":"2024-09-22T03:26:46.763012+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":107,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768173+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:46.768173+0000","last_peered":"2024-09-22T03:26:46.768173+0000","last_clean":"2024-09-22T03:26:46.768173+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:46.768173+0000","last_undegraded":"2024-09-22T03:26:46.768173+0000","last_fullsized":"2024-09-22T03:26:46.768173+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"45'1134","reported_seq":1191,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768965+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:46.768965+0000","last_peered":"2024-09-22T03:26:46.768965+0000","last_clean":"2024-09-22T03:26:46.768965+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:46.768965+0000","last_undegraded":"2024-09-22T03:26:46.768965+0000","last_fullsized":"2024-09-22T03:26:46.768965+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1134,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769371+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:46.769371+0000","last_peered":"2024-09-22T03:26:46.769371+0000","last_clean":"2024-09-22T03:26:46.769371+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:46.769371+0000","last_undegraded":"2024-09-22T03:26:46.769371+0000","last_fullsized":"2024-09-22T03:26:46.769371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"45'1170","reported_seq":1227,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766807+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:46.766807+0000","last_peered":"2024-09-22T03:26:46.766807+0000","last_clean":"2024-09-22T03:26:46.766807+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:46.766807+0000","last_undegraded":"2024-09-22T03:26:46.766807+0000","last_fullsized":"2024-09-22T03:26:46.766807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1170,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.764957+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:46.764957+0000","last_peered":"2024-09-22T03:26:46.764957+0000","last_clean":"2024-09-22T03:26:46.764957+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:46.764957+0000","last_undegraded":"2024-09-22T03:26:46.764957+0000","last_fullsized":"2024-09-22T03:26:46.764957+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"45'1071","reported_seq":1128,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.764071+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:46.764071+0000","last_peered":"2024-09-22T03:26:46.764071+0000","last_clean":"2024-09-22T03:26:46.764071+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:46.764071+0000","last_undegraded":"2024-09-22T03:26:46.764071+0000","last_fullsized":"2024-09-22T03:26:46.764071+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1071,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":108,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.765654+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:46.765654+0000","last_peered":"2024-09-22T03:26:46.765654+0000","last_clean":"2024-09-22T03:26:46.765654+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:46.765654+0000","last_undegraded":"2024-09-22T03:26:46.765654+0000","last_fullsized":"2024-09-22T03:26:46.765654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"45'1107","reported_seq":1164,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769412+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:46.769412+0000","last_peered":"2024-09-22T03:26:46.769412+0000","last_clean":"2024-09-22T03:26:46.769412+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:46.769412+0000","last_undegraded":"2024-09-22T03:26:46.769412+0000","last_fullsized":"2024-09-22T03:26:46.769412+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1107,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766283+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:46.766283+0000","last_peered":"2024-09-22T03:26:46.766283+0000","last_clean":"2024-09-22T03:26:46.766283+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:46.766283+0000","last_undegraded":"2024-09-22T03:26:46.766283+0000","last_fullsized":"2024-09-22T03:26:46.766283+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"45'1179","reported_seq":1236,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768356+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:46.768356+0000","last_peered":"2024-09-22T03:26:46.768356+0000","last_clean":"2024-09-22T03:26:46.768356+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:46.768356+0000","last_undegraded":"2024-09-22T03:26:46.768356+0000","last_fullsized":"2024-09-22T03:26:46.768356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1179,"log_dups_size":0,"ondisk_log_size":1179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1179,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766331+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:46.766331+0000","last_peered":"2024-09-22T03:26:46.766331+0000","last_clean":"2024-09-22T03:26:46.766331+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:46.766331+0000","last_undegraded":"2024-09-22T03:26:46.766331+0000","last_fullsized":"2024-09-22T03:26:46.766331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"45'1224","reported_seq":1281,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.767552+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:46.767552+0000","last_peered":"2024-09-22T03:26:46.767552+0000","last_clean":"2024-09-22T03:26:46.767552+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:46.767552+0000","last_undegraded":"2024-09-22T03:26:46.767552+0000","last_fullsized":"2024-09-22T03:26:46.767552+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1224,"log_dups_size":0,"ondisk_log_size":1224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1224,"num_write_kb":8704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766791+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:46.766791+0000","last_peered":"2024-09-22T03:26:46.766791+0000","last_clean":"2024-09-22T03:26:46.766791+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:46.766791+0000","last_undegraded":"2024-09-22T03:26:46.766791+0000","last_fullsized":"2024-09-22T03:26:46.766791+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"45'1008","reported_seq":1065,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.770317+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:46.770317+0000","last_peered":"2024-09-22T03:26:46.770317+0000","last_clean":"2024-09-22T03:26:46.770317+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:46.770317+0000","last_undegraded":"2024-09-22T03:26:46.770317+0000","last_fullsized":"2024-09-22T03:26:46.770317+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769951+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:46.769951+0000","last_peered":"2024-09-22T03:26:46.769951+0000","last_clean":"2024-09-22T03:26:46.769951+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:46.769951+0000","last_undegraded":"2024-09-22T03:26:46.769951+0000","last_fullsized":"2024-09-22T03:26:46.769951+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17926,"num_write_kb":127457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214176976896,"internally_reserved":0,"allocated":34516992,"data_stored":34516992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17926,"ondisk_log_size":17926,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214176976896,"internally_reserved":0,"allocated":34516992,"data_stored":34516992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":9044,"kb_used_data":9044,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52288684,"statfs":{"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7656,"kb_used_data":7656,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290072,"statfs":{"total":53552873472,"available":53545033728,"internally_reserved":0,"allocated":7839744,"data_stored":7839744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8248,"kb_used_data":8248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289480,"statfs":{"total":53552873472,"available":53544427520,"internally_reserved":0,"allocated":8445952,"data_stored":8445952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8760,"kb_used_data":8760,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52288968,"statfs":{"total":53552873472,"available":53543903232,"internally_reserved":0,"allocated":8970240,"data_stored":8970240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53543903232,"internally_reserved":0,"allocated":8970240,"data_stored":8970240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53544427520,"internally_reserved":0,"allocated":8445952,"data_stored":8445952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53545033728,"internally_reserved":0,"allocated":7839744,"data_stored":7839744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53543903232,"internally_reserved":0,"allocated":8970240,"data_stored":8970240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53544427520,"internally_reserved":0,"allocated":8445952,"data_stored":8445952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53545033728,"internally_reserved":0,"allocated":7839744,"data_stored":7839744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:48.082 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:48.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.180+0000 7f8cbc197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.187+0000 7f8cbc197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.361 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:26:48.362 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:26:48.373 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-22T03:26:47.976363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":18065,"num_write_kb":129353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18026,"ondisk_log_size":18026,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":33708,"kb_used_data":33708,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209157204,"statfs":{"total":214211493888,"available":214176976896,"internally_reserved":0,"allocated":34516992,"data_stored":34516992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47349784,"num_objects":-724,"num_object_clones":0,"num_object_copies":-2172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":724,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000678"},"pg_stats":[{"pgid":"4.f","version":"45'1305","reported_seq":1362,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.762834+0000","last_change":"2024-09-22T03:25:03.167733+0000","last_active":"2024-09-22T03:26:46.762834+0000","last_peered":"2024-09-22T03:26:46.762834+0000","last_clean":"2024-09-22T03:26:46.762834+0000","last_became_active":"2024-09-22T03:25:03.160064+0000","last_became_peered":"2024-09-22T03:25:03.160064+0000","last_unstale":"2024-09-22T03:26:46.762834+0000","last_undegraded":"2024-09-22T03:26:46.762834+0000","last_fullsized":"2024-09-22T03:26:46.762834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1305,"log_dups_size":0,"ondisk_log_size":1305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1305,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"45'972","reported_seq":1029,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.767089+0000","last_change":"2024-09-22T03:25:03.124380+0000","last_active":"2024-09-22T03:26:46.767089+0000","last_peered":"2024-09-22T03:26:46.767089+0000","last_clean":"2024-09-22T03:26:46.767089+0000","last_became_active":"2024-09-22T03:25:03.120932+0000","last_became_peered":"2024-09-22T03:25:03.120932+0000","last_unstale":"2024-09-22T03:26:46.767089+0000","last_undegraded":"2024-09-22T03:26:46.767089+0000","last_fullsized":"2024-09-22T03:26:46.767089+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":972,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"45'1100","reported_seq":1158,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.771258+0000","last_change":"2024-09-22T03:25:03.131798+0000","last_active":"2024-09-22T03:26:46.771258+0000","last_peered":"2024-09-22T03:26:46.771258+0000","last_clean":"2024-09-22T03:26:46.771258+0000","last_became_active":"2024-09-22T03:25:03.123516+0000","last_became_peered":"2024-09-22T03:25:03.123516+0000","last_unstale":"2024-09-22T03:26:46.771258+0000","last_undegraded":"2024-09-22T03:26:46.771258+0000","last_fullsized":"2024-09-22T03:26:46.771258+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1100,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"45'1053","reported_seq":1110,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.764638+0000","last_change":"2024-09-22T03:25:03.157776+0000","last_active":"2024-09-22T03:26:46.764638+0000","last_peered":"2024-09-22T03:26:46.764638+0000","last_clean":"2024-09-22T03:26:46.764638+0000","last_became_active":"2024-09-22T03:25:03.139180+0000","last_became_peered":"2024-09-22T03:25:03.139180+0000","last_unstale":"2024-09-22T03:26:46.764638+0000","last_undegraded":"2024-09-22T03:26:46.764638+0000","last_fullsized":"2024-09-22T03:26:46.764638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1053,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"45'1071","reported_seq":1128,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769783+0000","last_change":"2024-09-22T03:25:03.149361+0000","last_active":"2024-09-22T03:26:46.769783+0000","last_peered":"2024-09-22T03:26:46.769783+0000","last_clean":"2024-09-22T03:26:46.769783+0000","last_became_active":"2024-09-22T03:25:03.140988+0000","last_became_peered":"2024-09-22T03:25:03.140988+0000","last_unstale":"2024-09-22T03:26:46.769783+0000","last_undegraded":"2024-09-22T03:26:46.769783+0000","last_fullsized":"2024-09-22T03:26:46.769783+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1071,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"45'1161","reported_seq":1218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768388+0000","last_change":"2024-09-22T03:25:03.165581+0000","last_active":"2024-09-22T03:26:46.768388+0000","last_peered":"2024-09-22T03:26:46.768388+0000","last_clean":"2024-09-22T03:26:46.768388+0000","last_became_active":"2024-09-22T03:25:03.159865+0000","last_became_peered":"2024-09-22T03:25:03.159865+0000","last_unstale":"2024-09-22T03:26:46.768388+0000","last_undegraded":"2024-09-22T03:26:46.768388+0000","last_fullsized":"2024-09-22T03:26:46.768388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1161,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"45'1053","reported_seq":1110,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.771800+0000","last_change":"2024-09-22T03:25:03.183652+0000","last_active":"2024-09-22T03:26:46.771800+0000","last_peered":"2024-09-22T03:26:46.771800+0000","last_clean":"2024-09-22T03:26:46.771800+0000","last_became_active":"2024-09-22T03:25:03.177799+0000","last_became_peered":"2024-09-22T03:25:03.177799+0000","last_unstale":"2024-09-22T03:26:46.771800+0000","last_undegraded":"2024-09-22T03:26:46.771800+0000","last_fullsized":"2024-09-22T03:26:46.771800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1053,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"45'1175","reported_seq":1232,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.767220+0000","last_change":"2024-09-22T03:25:03.185957+0000","last_active":"2024-09-22T03:26:46.767220+0000","last_peered":"2024-09-22T03:26:46.767220+0000","last_clean":"2024-09-22T03:26:46.767220+0000","last_became_active":"2024-09-22T03:25:03.180273+0000","last_became_peered":"2024-09-22T03:25:03.180273+0000","last_unstale":"2024-09-22T03:26:46.767220+0000","last_undegraded":"2024-09-22T03:26:46.767220+0000","last_fullsized":"2024-09-22T03:26:46.767220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1175,"log_dups_size":0,"ondisk_log_size":1175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1175,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"45'1143","reported_seq":1200,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.765308+0000","last_change":"2024-09-22T03:25:03.188478+0000","last_active":"2024-09-22T03:26:46.765308+0000","last_peered":"2024-09-22T03:26:46.765308+0000","last_clean":"2024-09-22T03:26:46.765308+0000","last_became_active":"2024-09-22T03:25:03.182888+0000","last_became_peered":"2024-09-22T03:25:03.182888+0000","last_unstale":"2024-09-22T03:26:46.765308+0000","last_undegraded":"2024-09-22T03:26:46.765308+0000","last_fullsized":"2024-09-22T03:26:46.765308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1143,"log_dups_size":0,"ondisk_log_size":1143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1143,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":241,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.763012+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:46.763012+0000","last_peered":"2024-09-22T03:26:46.763012+0000","last_clean":"2024-09-22T03:26:46.763012+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:46.763012+0000","last_undegraded":"2024-09-22T03:26:46.763012+0000","last_fullsized":"2024-09-22T03:26:46.763012+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":107,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768173+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:46.768173+0000","last_peered":"2024-09-22T03:26:46.768173+0000","last_clean":"2024-09-22T03:26:46.768173+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:46.768173+0000","last_undegraded":"2024-09-22T03:26:46.768173+0000","last_fullsized":"2024-09-22T03:26:46.768173+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"45'1134","reported_seq":1191,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768965+0000","last_change":"2024-09-22T03:25:03.125615+0000","last_active":"2024-09-22T03:26:46.768965+0000","last_peered":"2024-09-22T03:26:46.768965+0000","last_clean":"2024-09-22T03:26:46.768965+0000","last_became_active":"2024-09-22T03:25:03.119218+0000","last_became_peered":"2024-09-22T03:25:03.119218+0000","last_unstale":"2024-09-22T03:26:46.768965+0000","last_undegraded":"2024-09-22T03:26:46.768965+0000","last_fullsized":"2024-09-22T03:26:46.768965+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1134,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769371+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:46.769371+0000","last_peered":"2024-09-22T03:26:46.769371+0000","last_clean":"2024-09-22T03:26:46.769371+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:46.769371+0000","last_undegraded":"2024-09-22T03:26:46.769371+0000","last_fullsized":"2024-09-22T03:26:46.769371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"45'1170","reported_seq":1227,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766807+0000","last_change":"2024-09-22T03:25:03.166201+0000","last_active":"2024-09-22T03:26:46.766807+0000","last_peered":"2024-09-22T03:26:46.766807+0000","last_clean":"2024-09-22T03:26:46.766807+0000","last_became_active":"2024-09-22T03:25:03.157582+0000","last_became_peered":"2024-09-22T03:25:03.157582+0000","last_unstale":"2024-09-22T03:26:46.766807+0000","last_undegraded":"2024-09-22T03:26:46.766807+0000","last_fullsized":"2024-09-22T03:26:46.766807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1170,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.764957+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:46.764957+0000","last_peered":"2024-09-22T03:26:46.764957+0000","last_clean":"2024-09-22T03:26:46.764957+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:46.764957+0000","last_undegraded":"2024-09-22T03:26:46.764957+0000","last_fullsized":"2024-09-22T03:26:46.764957+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"45'1071","reported_seq":1128,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.764071+0000","last_change":"2024-09-22T03:25:03.132239+0000","last_active":"2024-09-22T03:26:46.764071+0000","last_peered":"2024-09-22T03:26:46.764071+0000","last_clean":"2024-09-22T03:26:46.764071+0000","last_became_active":"2024-09-22T03:25:03.120036+0000","last_became_peered":"2024-09-22T03:25:03.120036+0000","last_unstale":"2024-09-22T03:26:46.764071+0000","last_undegraded":"2024-09-22T03:26:46.764071+0000","last_fullsized":"2024-09-22T03:26:46.764071+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1071,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":108,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.765654+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:46.765654+0000","last_peered":"2024-09-22T03:26:46.765654+0000","last_clean":"2024-09-22T03:26:46.765654+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:46.765654+0000","last_undegraded":"2024-09-22T03:26:46.765654+0000","last_fullsized":"2024-09-22T03:26:46.765654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"45'1107","reported_seq":1164,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769412+0000","last_change":"2024-09-22T03:25:03.142289+0000","last_active":"2024-09-22T03:26:46.769412+0000","last_peered":"2024-09-22T03:26:46.769412+0000","last_clean":"2024-09-22T03:26:46.769412+0000","last_became_active":"2024-09-22T03:25:03.135764+0000","last_became_peered":"2024-09-22T03:25:03.135764+0000","last_unstale":"2024-09-22T03:26:46.769412+0000","last_undegraded":"2024-09-22T03:26:46.769412+0000","last_fullsized":"2024-09-22T03:26:46.769412+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1107,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766283+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:46.766283+0000","last_peered":"2024-09-22T03:26:46.766283+0000","last_clean":"2024-09-22T03:26:46.766283+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:46.766283+0000","last_undegraded":"2024-09-22T03:26:46.766283+0000","last_fullsized":"2024-09-22T03:26:46.766283+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"45'1179","reported_seq":1236,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.768356+0000","last_change":"2024-09-22T03:25:03.114844+0000","last_active":"2024-09-22T03:26:46.768356+0000","last_peered":"2024-09-22T03:26:46.768356+0000","last_clean":"2024-09-22T03:26:46.768356+0000","last_became_active":"2024-09-22T03:25:03.111397+0000","last_became_peered":"2024-09-22T03:25:03.111397+0000","last_unstale":"2024-09-22T03:26:46.768356+0000","last_undegraded":"2024-09-22T03:26:46.768356+0000","last_fullsized":"2024-09-22T03:26:46.768356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1179,"log_dups_size":0,"ondisk_log_size":1179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1179,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766331+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:46.766331+0000","last_peered":"2024-09-22T03:26:46.766331+0000","last_clean":"2024-09-22T03:26:46.766331+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:46.766331+0000","last_undegraded":"2024-09-22T03:26:46.766331+0000","last_fullsized":"2024-09-22T03:26:46.766331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"45'1224","reported_seq":1281,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.767552+0000","last_change":"2024-09-22T03:25:03.139005+0000","last_active":"2024-09-22T03:26:46.767552+0000","last_peered":"2024-09-22T03:26:46.767552+0000","last_clean":"2024-09-22T03:26:46.767552+0000","last_became_active":"2024-09-22T03:25:03.135476+0000","last_became_peered":"2024-09-22T03:25:03.135476+0000","last_unstale":"2024-09-22T03:26:46.767552+0000","last_undegraded":"2024-09-22T03:26:46.767552+0000","last_fullsized":"2024-09-22T03:26:46.767552+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1224,"log_dups_size":0,"ondisk_log_size":1224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1224,"num_write_kb":8704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.766791+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:46.766791+0000","last_peered":"2024-09-22T03:26:46.766791+0000","last_clean":"2024-09-22T03:26:46.766791+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:46.766791+0000","last_undegraded":"2024-09-22T03:26:46.766791+0000","last_fullsized":"2024-09-22T03:26:46.766791+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"45'1008","reported_seq":1065,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.770317+0000","last_change":"2024-09-22T03:25:03.175015+0000","last_active":"2024-09-22T03:26:46.770317+0000","last_peered":"2024-09-22T03:26:46.770317+0000","last_clean":"2024-09-22T03:26:46.770317+0000","last_became_active":"2024-09-22T03:25:03.157857+0000","last_became_peered":"2024-09-22T03:25:03.157857+0000","last_unstale":"2024-09-22T03:26:46.770317+0000","last_undegraded":"2024-09-22T03:26:46.770317+0000","last_fullsized":"2024-09-22T03:26:46.770317+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:25:02.019664+0000","last_clean_scrub_stamp":"2024-09-22T03:25:02.019664+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-22T03:26:46.769951+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:46.769951+0000","last_peered":"2024-09-22T03:26:46.769951+0000","last_clean":"2024-09-22T03:26:46.769951+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:46.769951+0000","last_undegraded":"2024-09-22T03:26:46.769951+0000","last_fullsized":"2024-09-22T03:26:46.769951+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17926,"num_write_kb":127457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214176976896,"internally_reserved":0,"allocated":34516992,"data_stored":34516992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17926,"ondisk_log_size":17926,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214176976896,"internally_reserved":0,"allocated":34516992,"data_stored":34516992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":9044,"kb_used_data":9044,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52288684,"statfs":{"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7656,"kb_used_data":7656,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290072,"statfs":{"total":53552873472,"available":53545033728,"internally_reserved":0,"allocated":7839744,"data_stored":7839744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8248,"kb_used_data":8248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289480,"statfs":{"total":53552873472,"available":53544427520,"internally_reserved":0,"allocated":8445952,"data_stored":8445952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8760,"kb_used_data":8760,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52288968,"statfs":{"total":53552873472,"available":53543903232,"internally_reserved":0,"allocated":8970240,"data_stored":8970240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53543903232,"internally_reserved":0,"allocated":8970240,"data_stored":8970240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53544427520,"internally_reserved":0,"allocated":8445952,"data_stored":8445952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53545033728,"internally_reserved":0,"allocated":7839744,"data_stored":7839744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":53552873472,"available":53543903232,"internally_reserved":0,"allocated":8970240,"data_stored":8970240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":53552873472,"available":53544427520,"internally_reserved":0,"allocated":8445952,"data_stored":8445952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":53552873472,"available":53545033728,"internally_reserved":0,"allocated":7839744,"data_stored":7839744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":53552873472,"available":53543612416,"internally_reserved":0,"allocated":9261056,"data_stored":9261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:26:48.375 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:26:48.375 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:48.375 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:48.375 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:48.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.474+0000 7fa57bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.481+0000 7fa57bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:48.496 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:48.496 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:48.498 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:48.499 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:48.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:48.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.598+0000 7fa9d064f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.605+0000 7fa9d064f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.619 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:48.619 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:48.619 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:48.620 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:48.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:48.623 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:48.623 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:48.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.722+0000 7f49c3418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.729+0000 7f49c3418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.745 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:48.745 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:48.745 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:48.745 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:48.745 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:48.746 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:48.746 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:48.746 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:48.746 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:48.746 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:48.746 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:48.746 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:48.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:48.748 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:48.749 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:48.758 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_1' removed 2024-09-22T03:26:48.780 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:26:48.781 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:26:48.781 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-22T03:26:48.781 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_2 16 2024-09-22T03:26:48.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.914+0000 7f5d8434e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.931+0000 7f5d8434e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.954+0000 7f261d797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:48.968+0000 7f261d797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:48.992 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:48.992 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:48.992 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:48.992 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:48.992 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:48.993 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:48.993 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:48.993 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:48.993 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:48.993 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:48.993 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:48.993 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:48.995 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:48.995 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:49.385 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:49.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:49.485+0000 7f1ac7117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:49.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:49.489+0000 7f1ac7117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:49.762 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_2' created 2024-09-22T03:26:49.780 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-22T03:26:49.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:49.898+0000 7f09f7597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:49.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:49.910+0000 7f09f7597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:50.765 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-22T03:26:51.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:51.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:51.672+0000 7ff4f4663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:51.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:51.677+0000 7ff4f4663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:51.785 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-22T03:26:51.787 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:26:51.787 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:26:51.841 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:26:51.841 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:26:51.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_39572 2024-09-22T03:26:51.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:26:51.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:26:52.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 192 176 1.37491 1.375 0.139619 0.0840807 2024-09-22T03:26:53.751 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:53.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 371 355 1.3866 1.39844 0.120802 0.0878979 2024-09-22T03:26:53.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:53.865+0000 7f6c62957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:53.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:53.871+0000 7f6c62957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:53.996 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:53.996 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:53.996 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:26:53.997 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:26:54.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.126+0000 7fddc9d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.137+0000 7fddc9d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:54.216 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:54.220 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:54.220 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:54.220 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:26:54.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.351+0000 7f38eb218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.362+0000 7f38eb218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:54.395 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:54.399 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:54.399 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:54.399 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:26:54.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.548+0000 7f1b54463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.559+0000 7f1b54463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:54.581 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:54.581 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:54.581 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:54.581 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:54.584 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:54.584 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:54.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:26:54.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.696+0000 7f4d2c8ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:54.705+0000 7f4d2c8ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:26:54.763 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:26:54.767 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:26:54.767 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:26:54.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 563 547 1.42436 1.5 0.0744245 0.0863385 2024-09-22T03:26:55.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 731 715 1.39637 1.3125 0.0535699 0.0882534 2024-09-22T03:26:56.038 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:26:56.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:56.146+0000 7fcafe62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:56.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:56.151+0000 7fcafe62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:56.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 889 873 1.36395 1.23438 0.0352731 0.0906255 2024-09-22T03:26:57.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 1077 1061 1.38139 1.46875 0.0335443 0.08947 2024-09-22T03:26:58.292 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:26:58.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:58.418+0000 7f7ce4997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:58.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:58.425+0000 7f7ce4997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:58.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 1243 1227 1.3693 1.29688 0.047465 0.0895457 2024-09-22T03:26:59.768 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:26:59.768 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:26:59.768 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:26:59.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 16 1439 1423 1.38953 1.53125 0.0633015 0.089405 2024-09-22T03:26:59.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:59.909+0000 7f6987b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:26:59.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:26:59.918+0000 7f6987b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:00.111 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:27:00.111 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:27:00.126 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-22T03:26:59.978605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8086067,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1054,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":999,"ondisk_log_size":999,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":53316,"kb_used_data":53316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209137596,"statfs":{"total":214211493888,"available":214156898304,"internally_reserved":0,"allocated":54595584,"data_stored":54595584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7495680,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":915,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.213004"},"pg_stats":[{"pgid":"5.f","version":"51'71","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.869749+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:26:56.869749+0000","last_peered":"2024-09-22T03:26:56.869749+0000","last_clean":"2024-09-22T03:26:56.869749+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:26:56.869749+0000","last_undegraded":"2024-09-22T03:26:56.869749+0000","last_fullsized":"2024-09-22T03:26:56.869749+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"51'87","reported_seq":100,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.930817+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:26:56.930817+0000","last_peered":"2024-09-22T03:26:56.930817+0000","last_clean":"2024-09-22T03:26:56.930817+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:26:56.930817+0000","last_undegraded":"2024-09-22T03:26:56.930817+0000","last_fullsized":"2024-09-22T03:26:56.930817+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"51'47","reported_seq":60,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.414211+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:26:56.414211+0000","last_peered":"2024-09-22T03:26:56.414211+0000","last_clean":"2024-09-22T03:26:56.414211+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:26:56.414211+0000","last_undegraded":"2024-09-22T03:26:56.414211+0000","last_fullsized":"2024-09-22T03:26:56.414211+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"51'31","reported_seq":44,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.564900+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:26:56.564900+0000","last_peered":"2024-09-22T03:26:56.564900+0000","last_clean":"2024-09-22T03:26:56.564900+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:26:56.564900+0000","last_undegraded":"2024-09-22T03:26:56.564900+0000","last_fullsized":"2024-09-22T03:26:56.564900+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"51'47","reported_seq":60,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.530847+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:26:56.530847+0000","last_peered":"2024-09-22T03:26:56.530847+0000","last_clean":"2024-09-22T03:26:56.530847+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:26:56.530847+0000","last_undegraded":"2024-09-22T03:26:56.530847+0000","last_fullsized":"2024-09-22T03:26:56.530847+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"51'55","reported_seq":68,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.170974+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:26:56.170974+0000","last_peered":"2024-09-22T03:26:56.170974+0000","last_clean":"2024-09-22T03:26:56.170974+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:26:56.170974+0000","last_undegraded":"2024-09-22T03:26:56.170974+0000","last_fullsized":"2024-09-22T03:26:56.170974+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"51'47","reported_seq":60,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:54.766098+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:26:54.766098+0000","last_peered":"2024-09-22T03:26:54.766098+0000","last_clean":"2024-09-22T03:26:54.766098+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:26:54.766098+0000","last_undegraded":"2024-09-22T03:26:54.766098+0000","last_fullsized":"2024-09-22T03:26:54.766098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"51'47","reported_seq":60,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.836754+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:26:56.836754+0000","last_peered":"2024-09-22T03:26:56.836754+0000","last_clean":"2024-09-22T03:26:56.836754+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:26:56.836754+0000","last_undegraded":"2024-09-22T03:26:56.836754+0000","last_fullsized":"2024-09-22T03:26:56.836754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"51'63","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.766087+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:26:56.766087+0000","last_peered":"2024-09-22T03:26:56.766087+0000","last_clean":"2024-09-22T03:26:56.766087+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:26:56.766087+0000","last_undegraded":"2024-09-22T03:26:56.766087+0000","last_fullsized":"2024-09-22T03:26:56.766087+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":256,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.781725+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:26:51.781725+0000","last_peered":"2024-09-22T03:26:51.781725+0000","last_clean":"2024-09-22T03:26:51.781725+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:26:51.781725+0000","last_undegraded":"2024-09-22T03:26:51.781725+0000","last_fullsized":"2024-09-22T03:26:51.781725+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.781015+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:26:51.781015+0000","last_peered":"2024-09-22T03:26:51.781015+0000","last_clean":"2024-09-22T03:26:51.781015+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:26:51.781015+0000","last_undegraded":"2024-09-22T03:26:51.781015+0000","last_fullsized":"2024-09-22T03:26:51.781015+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"51'39","reported_seq":52,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:55.610905+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:26:55.610905+0000","last_peered":"2024-09-22T03:26:55.610905+0000","last_clean":"2024-09-22T03:26:55.610905+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:26:55.610905+0000","last_undegraded":"2024-09-22T03:26:55.610905+0000","last_fullsized":"2024-09-22T03:26:55.610905+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.784437+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:26:51.784437+0000","last_peered":"2024-09-22T03:26:51.784437+0000","last_clean":"2024-09-22T03:26:51.784437+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:26:51.784437+0000","last_undegraded":"2024-09-22T03:26:51.784437+0000","last_fullsized":"2024-09-22T03:26:51.784437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"51'71","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.919517+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:26:56.919517+0000","last_peered":"2024-09-22T03:26:56.919517+0000","last_clean":"2024-09-22T03:26:56.919517+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:26:56.919517+0000","last_undegraded":"2024-09-22T03:26:56.919517+0000","last_fullsized":"2024-09-22T03:26:56.919517+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.779920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:51.779920+0000","last_peered":"2024-09-22T03:26:51.779920+0000","last_clean":"2024-09-22T03:26:51.779920+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:26:51.779920+0000","last_undegraded":"2024-09-22T03:26:51.779920+0000","last_fullsized":"2024-09-22T03:26:51.779920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"51'55","reported_seq":68,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.736954+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:26:56.736954+0000","last_peered":"2024-09-22T03:26:56.736954+0000","last_clean":"2024-09-22T03:26:56.736954+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:26:56.736954+0000","last_undegraded":"2024-09-22T03:26:56.736954+0000","last_fullsized":"2024-09-22T03:26:56.736954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":123,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.780196+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:26:51.780196+0000","last_peered":"2024-09-22T03:26:51.780196+0000","last_clean":"2024-09-22T03:26:51.780196+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:26:51.780196+0000","last_undegraded":"2024-09-22T03:26:51.780196+0000","last_fullsized":"2024-09-22T03:26:51.780196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"51'71","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.352547+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:26:56.352547+0000","last_peered":"2024-09-22T03:26:56.352547+0000","last_clean":"2024-09-22T03:26:56.352547+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:26:56.352547+0000","last_undegraded":"2024-09-22T03:26:56.352547+0000","last_fullsized":"2024-09-22T03:26:56.352547+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.785217+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:51.785217+0000","last_peered":"2024-09-22T03:26:51.785217+0000","last_clean":"2024-09-22T03:26:51.785217+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:26:51.785217+0000","last_undegraded":"2024-09-22T03:26:51.785217+0000","last_fullsized":"2024-09-22T03:26:51.785217+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"51'39","reported_seq":52,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:55.927557+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:26:55.927557+0000","last_peered":"2024-09-22T03:26:55.927557+0000","last_clean":"2024-09-22T03:26:55.927557+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:26:55.927557+0000","last_undegraded":"2024-09-22T03:26:55.927557+0000","last_fullsized":"2024-09-22T03:26:55.927557+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.785010+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:26:51.785010+0000","last_peered":"2024-09-22T03:26:51.785010+0000","last_clean":"2024-09-22T03:26:51.785010+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:26:51.785010+0000","last_undegraded":"2024-09-22T03:26:51.785010+0000","last_fullsized":"2024-09-22T03:26:51.785010+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"51'58","reported_seq":71,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.946151+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:26:56.946151+0000","last_peered":"2024-09-22T03:26:56.946151+0000","last_clean":"2024-09-22T03:26:56.946151+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:26:56.946151+0000","last_undegraded":"2024-09-22T03:26:56.946151+0000","last_fullsized":"2024-09-22T03:26:56.946151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483328,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.787257+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:26:51.787257+0000","last_peered":"2024-09-22T03:26:51.787257+0000","last_clean":"2024-09-22T03:26:51.787257+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:26:51.787257+0000","last_undegraded":"2024-09-22T03:26:51.787257+0000","last_fullsized":"2024-09-22T03:26:51.787257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"51'71","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:56.581866+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:26:56.581866+0000","last_peered":"2024-09-22T03:26:56.581866+0000","last_clean":"2024-09-22T03:26:56.581866+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:26:56.581866+0000","last_undegraded":"2024-09-22T03:26:56.581866+0000","last_fullsized":"2024-09-22T03:26:56.581866+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-22T03:26:51.781501+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:26:51.781501+0000","last_peered":"2024-09-22T03:26:51.781501+0000","last_clean":"2024-09-22T03:26:51.781501+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:26:51.781501+0000","last_undegraded":"2024-09-22T03:26:51.781501+0000","last_fullsized":"2024-09-22T03:26:51.781501+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":7495680,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":915,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214156603392,"internally_reserved":0,"allocated":54890496,"data_stored":54890496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":899,"ondisk_log_size":899,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214156619776,"internally_reserved":0,"allocated":54874112,"data_stored":54874112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53539905536,"internally_reserved":0,"allocated":12967936,"data_stored":12967936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":12528,"kb_used_data":12528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52285200,"statfs":{"total":53552873472,"available":53540044800,"internally_reserved":0,"allocated":12828672,"data_stored":12828672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":15944,"kb_used_data":15944,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52281784,"statfs":{"total":53552873472,"available":53536546816,"internally_reserved":0,"allocated":16326656,"data_stored":16326656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":10024,"kb_used_data":10024,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52287704,"statfs":{"total":53552873472,"available":53542608896,"internally_reserved":0,"allocated":10264576,"data_stored":10264576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":14820,"kb_used_data":14820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52282908,"statfs":{"total":53552873472,"available":53537697792,"internally_reserved":0,"allocated":15175680,"data_stored":15175680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53539905536,"internally_reserved":0,"allocated":12967936,"data_stored":12967936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53537689600,"internally_reserved":0,"allocated":15183872,"data_stored":15183872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53542608896,"internally_reserved":0,"allocated":10264576,"data_stored":10264576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53536423936,"internally_reserved":0,"allocated":16449536,"data_stored":16449536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53539897344,"internally_reserved":0,"allocated":12976128,"data_stored":12976128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53537689600,"internally_reserved":0,"allocated":15183872,"data_stored":15183872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53542608896,"internally_reserved":0,"allocated":10264576,"data_stored":10264576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53536415744,"internally_reserved":0,"allocated":16457728,"data_stored":16457728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53539889152,"internally_reserved":0,"allocated":12984320,"data_stored":12984320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:00.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:00.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:00.665+0000 7f83d710e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:00.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:00.673+0000 7f83d710e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:00.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1623 1607 1.39484 1.4375 0.0314178 0.0891069 2024-09-22T03:27:01.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 1796 1780 1.3905 1.35156 0.183873 0.0893424 2024-09-22T03:27:02.842 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 1992 1976 1.40329 1.53125 0.123424 0.0887007 2024-09-22T03:27:02.889 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:03.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:02.999+0000 7f223a717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:03.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:03.005+0000 7f223a717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:03.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 2197 2181 1.4198 1.60156 0.0874382 0.087664 2024-09-22T03:27:04.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 2380 2364 1.42055 1.42969 0.0465805 0.0874834 2024-09-22T03:27:05.128 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:05.129 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:05.129 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:27:05.129 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:27:05.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:05.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.253+0000 7f4946957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.263+0000 7f4946957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.280 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:05.280 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:05.280 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:05.280 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:05.280 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:05.280 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:05.280 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:05.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:05.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:05.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:05.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:05.281 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:05.285 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:05.285 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:05.286 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:27:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.357+0000 7f299d917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.362+0000 7f299d917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.452+0000 7fd1a6717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.462+0000 7fd1a6717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.529 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:05.530 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:05.531 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:05.533 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:05.534 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:05.534 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:27:05.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.696+0000 7fb161a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.706+0000 7fb161a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:05.722 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:05.723 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:05.723 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:05.726 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:05.726 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:05.726 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:27:05.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.835+0000 7fbcbbf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 2524 2508 1.39943 1.125 0.0433069 0.0886759 2024-09-22T03:27:05.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:05.844+0000 7fbcbbf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:05.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:05.893 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:05.893 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:05.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:05.896 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:06.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 2713 2697 1.40457 1.47656 0.0687483 0.0888077 2024-09-22T03:27:07.515 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:07.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:07.682+0000 7f911fac3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:07.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:07.688+0000 7f911fac3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:07.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 2878 2862 1.39734 1.28906 0.0512589 0.0891832 2024-09-22T03:27:08.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 3076 3060 1.40613 1.54688 0.125498 0.0886028 2024-09-22T03:27:09.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 3252 3236 1.40439 1.375 0.13414 0.0887275 2024-09-22T03:27:09.866 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:09.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:09.977+0000 7f00acc2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:09.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:09.983+0000 7f00acc2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:10.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 3427 3411 1.40243 1.36719 0.0943647 0.088841 2024-09-22T03:27:10.896 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:10.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:27:11.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:11.023+0000 7f6021eba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:11.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:11.033+0000 7f6021eba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:11.710 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:27:11.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:27:11.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:27:11.843051+0000 min lat: 0.0117735 max lat: 0.474035 avg lat: 0.0889558 2024-09-22T03:27:11.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:27:11.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 3600 3584 1.39988 1.35156 0.0495731 0.0889558 2024-09-22T03:27:11.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:11.862+0000 7fae2bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:11.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:11.874+0000 7fae2bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:12.185 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:12.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:12.339+0000 7fc87ecd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:12.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:12.344+0000 7fc87ecd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:12.715 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:27:12.733 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:27:12.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 3744 3728 1.38678 1.125 0.292882 0.0895967 2024-09-22T03:27:12.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:12.847+0000 7f2ead197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:12.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:12.862+0000 7f2ead197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:13.722 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:27:13.741 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:27:13.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 3919 3903 1.38589 1.36719 0.0460843 0.0899153 2024-09-22T03:27:13.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:13.854+0000 7fc80b658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:13.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:13.865+0000 7fc80b658640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.445 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:14.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:14.596+0000 7f1d648d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:14.601+0000 7f1d648d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:27:14.750 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:27:14.751 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:14.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 4086 4070 1.38235 1.30469 0.270289 0.0900096 2024-09-22T03:27:14.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:14.930+0000 7f70f402c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:14.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:14.939+0000 7f70f402c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.240 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:27:15.241 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:27:15.265 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-22T03:27:14.727210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30212659,"num_objects":456,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3755,"num_write_kb":30824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3713,"ondisk_log_size":3713,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":183220,"kb_used_data":183220,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209007692,"statfs":{"total":214211493888,"available":214023876608,"internally_reserved":0,"allocated":187617280,"data_stored":187617280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7364608,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":899,"num_write_kb":7192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746810"},"pg_stats":[{"pgid":"5.f","version":"51'144","reported_seq":159,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.733600+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:27:11.733600+0000","last_peered":"2024-09-22T03:27:11.733600+0000","last_clean":"2024-09-22T03:27:11.733600+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:27:11.733600+0000","last_undegraded":"2024-09-22T03:27:11.733600+0000","last_fullsized":"2024-09-22T03:27:11.733600+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"51'184","reported_seq":199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.738698+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:27:11.738698+0000","last_peered":"2024-09-22T03:27:11.738698+0000","last_clean":"2024-09-22T03:27:11.738698+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:27:11.738698+0000","last_undegraded":"2024-09-22T03:27:11.738698+0000","last_fullsized":"2024-09-22T03:27:11.738698+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"51'240","reported_seq":255,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.733847+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:27:11.733847+0000","last_peered":"2024-09-22T03:27:11.733847+0000","last_clean":"2024-09-22T03:27:11.733847+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:27:11.733847+0000","last_undegraded":"2024-09-22T03:27:11.733847+0000","last_fullsized":"2024-09-22T03:27:11.733847+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"51'192","reported_seq":207,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.729817+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:27:11.729817+0000","last_peered":"2024-09-22T03:27:11.729817+0000","last_clean":"2024-09-22T03:27:11.729817+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:27:11.729817+0000","last_undegraded":"2024-09-22T03:27:11.729817+0000","last_fullsized":"2024-09-22T03:27:11.729817+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"51'168","reported_seq":183,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.738313+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:27:11.738313+0000","last_peered":"2024-09-22T03:27:11.738313+0000","last_clean":"2024-09-22T03:27:11.738313+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:27:11.738313+0000","last_undegraded":"2024-09-22T03:27:11.738313+0000","last_fullsized":"2024-09-22T03:27:11.738313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"51'176","reported_seq":191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.734100+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:27:11.734100+0000","last_peered":"2024-09-22T03:27:11.734100+0000","last_clean":"2024-09-22T03:27:11.734100+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:27:11.734100+0000","last_undegraded":"2024-09-22T03:27:11.734100+0000","last_fullsized":"2024-09-22T03:27:11.734100+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"51'208","reported_seq":223,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735863+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:27:11.735863+0000","last_peered":"2024-09-22T03:27:11.735863+0000","last_clean":"2024-09-22T03:27:11.735863+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:27:11.735863+0000","last_undegraded":"2024-09-22T03:27:11.735863+0000","last_fullsized":"2024-09-22T03:27:11.735863+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"52'183","reported_seq":199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.802315+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:27:11.802315+0000","last_peered":"2024-09-22T03:27:11.802315+0000","last_clean":"2024-09-22T03:27:11.802315+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:27:11.802315+0000","last_undegraded":"2024-09-22T03:27:11.802315+0000","last_fullsized":"2024-09-22T03:27:11.802315+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"52'287","reported_seq":303,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.894548+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:27:11.894548+0000","last_peered":"2024-09-22T03:27:11.894548+0000","last_clean":"2024-09-22T03:27:11.894548+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:27:11.894548+0000","last_undegraded":"2024-09-22T03:27:11.894548+0000","last_fullsized":"2024-09-22T03:27:11.894548+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":259,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.727525+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:11.727525+0000","last_peered":"2024-09-22T03:27:11.727525+0000","last_clean":"2024-09-22T03:27:11.727525+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:11.727525+0000","last_undegraded":"2024-09-22T03:27:11.727525+0000","last_fullsized":"2024-09-22T03:27:11.727525+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":125,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.736450+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:11.736450+0000","last_peered":"2024-09-22T03:27:11.736450+0000","last_clean":"2024-09-22T03:27:11.736450+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:11.736450+0000","last_undegraded":"2024-09-22T03:27:11.736450+0000","last_fullsized":"2024-09-22T03:27:11.736450+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"51'216","reported_seq":231,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.740555+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:27:11.740555+0000","last_peered":"2024-09-22T03:27:11.740555+0000","last_clean":"2024-09-22T03:27:11.740555+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:27:11.740555+0000","last_undegraded":"2024-09-22T03:27:11.740555+0000","last_fullsized":"2024-09-22T03:27:11.740555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.740328+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:11.740328+0000","last_peered":"2024-09-22T03:27:11.740328+0000","last_clean":"2024-09-22T03:27:11.740328+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:11.740328+0000","last_undegraded":"2024-09-22T03:27:11.740328+0000","last_fullsized":"2024-09-22T03:27:11.740328+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"51'280","reported_seq":295,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.738189+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:27:11.738189+0000","last_peered":"2024-09-22T03:27:11.738189+0000","last_clean":"2024-09-22T03:27:11.738189+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:27:11.738189+0000","last_undegraded":"2024-09-22T03:27:11.738189+0000","last_fullsized":"2024-09-22T03:27:11.738189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.733568+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:11.733568+0000","last_peered":"2024-09-22T03:27:11.733568+0000","last_clean":"2024-09-22T03:27:11.733568+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:11.733568+0000","last_undegraded":"2024-09-22T03:27:11.733568+0000","last_fullsized":"2024-09-22T03:27:11.733568+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"51'328","reported_seq":343,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735150+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:27:11.735150+0000","last_peered":"2024-09-22T03:27:11.735150+0000","last_clean":"2024-09-22T03:27:11.735150+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:27:11.735150+0000","last_undegraded":"2024-09-22T03:27:11.735150+0000","last_fullsized":"2024-09-22T03:27:11.735150+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":126,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735858+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:11.735858+0000","last_peered":"2024-09-22T03:27:11.735858+0000","last_clean":"2024-09-22T03:27:11.735858+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:11.735858+0000","last_undegraded":"2024-09-22T03:27:11.735858+0000","last_fullsized":"2024-09-22T03:27:11.735858+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"51'216","reported_seq":231,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.732812+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:27:11.732812+0000","last_peered":"2024-09-22T03:27:11.732812+0000","last_clean":"2024-09-22T03:27:11.732812+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:27:11.732812+0000","last_undegraded":"2024-09-22T03:27:11.732812+0000","last_fullsized":"2024-09-22T03:27:11.732812+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.736987+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:11.736987+0000","last_peered":"2024-09-22T03:27:11.736987+0000","last_clean":"2024-09-22T03:27:11.736987+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:11.736987+0000","last_undegraded":"2024-09-22T03:27:11.736987+0000","last_fullsized":"2024-09-22T03:27:11.736987+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"52'295","reported_seq":311,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.947740+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:27:11.947740+0000","last_peered":"2024-09-22T03:27:11.947740+0000","last_clean":"2024-09-22T03:27:11.947740+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:27:11.947740+0000","last_undegraded":"2024-09-22T03:27:11.947740+0000","last_fullsized":"2024-09-22T03:27:11.947740+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.730826+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:11.730826+0000","last_peered":"2024-09-22T03:27:11.730826+0000","last_clean":"2024-09-22T03:27:11.730826+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:11.730826+0000","last_undegraded":"2024-09-22T03:27:11.730826+0000","last_fullsized":"2024-09-22T03:27:11.730826+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"51'248","reported_seq":263,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.729698+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:27:11.729698+0000","last_peered":"2024-09-22T03:27:11.729698+0000","last_clean":"2024-09-22T03:27:11.729698+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:27:11.729698+0000","last_undegraded":"2024-09-22T03:27:11.729698+0000","last_fullsized":"2024-09-22T03:27:11.729698+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735914+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:11.735914+0000","last_peered":"2024-09-22T03:27:11.735914+0000","last_clean":"2024-09-22T03:27:11.735914+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:11.735914+0000","last_undegraded":"2024-09-22T03:27:11.735914+0000","last_fullsized":"2024-09-22T03:27:11.735914+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"51'248","reported_seq":263,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.739419+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:27:11.739419+0000","last_peered":"2024-09-22T03:27:11.739419+0000","last_clean":"2024-09-22T03:27:11.739419+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:27:11.739419+0000","last_undegraded":"2024-09-22T03:27:11.739419+0000","last_fullsized":"2024-09-22T03:27:11.739419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.736233+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:11.736233+0000","last_peered":"2024-09-22T03:27:11.736233+0000","last_clean":"2024-09-22T03:27:11.736233+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:11.736233+0000","last_undegraded":"2024-09-22T03:27:11.736233+0000","last_fullsized":"2024-09-22T03:27:11.736233+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":29622272,"num_objects":452,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214021099520,"internally_reserved":0,"allocated":190394368,"data_stored":190394368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3613,"ondisk_log_size":3613,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214021099520,"internally_reserved":0,"allocated":190394368,"data_stored":190394368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":42580,"kb_used_data":42580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52255148,"statfs":{"total":53552873472,"available":53509271552,"internally_reserved":0,"allocated":43601920,"data_stored":43601920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":54080,"kb_used_data":54080,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52243648,"statfs":{"total":53552873472,"available":53497495552,"internally_reserved":0,"allocated":55377920,"data_stored":55377920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":39612,"kb_used_data":39612,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52258116,"statfs":{"total":53552873472,"available":53512310784,"internally_reserved":0,"allocated":40562688,"data_stored":40562688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":46948,"kb_used_data":46948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52250780,"statfs":{"total":53552873472,"available":53504798720,"internally_reserved":0,"allocated":48074752,"data_stored":48074752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53502263296,"internally_reserved":0,"allocated":50610176,"data_stored":50610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53512196096,"internally_reserved":0,"allocated":40677376,"data_stored":40677376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53497487360,"internally_reserved":0,"allocated":55386112,"data_stored":55386112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53502263296,"internally_reserved":0,"allocated":50610176,"data_stored":50610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53512196096,"internally_reserved":0,"allocated":40677376,"data_stored":40677376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53497487360,"internally_reserved":0,"allocated":55386112,"data_stored":55386112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:15.266 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:15.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:15.397+0000 7f93e7f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:15.407+0000 7f93e7f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:27:15.623 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:27:15.643 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-22T03:27:14.727210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30212659,"num_objects":456,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3755,"num_write_kb":30824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3713,"ondisk_log_size":3713,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":183220,"kb_used_data":183220,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209007692,"statfs":{"total":214211493888,"available":214023876608,"internally_reserved":0,"allocated":187617280,"data_stored":187617280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7364608,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":899,"num_write_kb":7192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746810"},"pg_stats":[{"pgid":"5.f","version":"51'144","reported_seq":159,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.733600+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:27:11.733600+0000","last_peered":"2024-09-22T03:27:11.733600+0000","last_clean":"2024-09-22T03:27:11.733600+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:27:11.733600+0000","last_undegraded":"2024-09-22T03:27:11.733600+0000","last_fullsized":"2024-09-22T03:27:11.733600+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"51'184","reported_seq":199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.738698+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:27:11.738698+0000","last_peered":"2024-09-22T03:27:11.738698+0000","last_clean":"2024-09-22T03:27:11.738698+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:27:11.738698+0000","last_undegraded":"2024-09-22T03:27:11.738698+0000","last_fullsized":"2024-09-22T03:27:11.738698+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"51'240","reported_seq":255,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.733847+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:27:11.733847+0000","last_peered":"2024-09-22T03:27:11.733847+0000","last_clean":"2024-09-22T03:27:11.733847+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:27:11.733847+0000","last_undegraded":"2024-09-22T03:27:11.733847+0000","last_fullsized":"2024-09-22T03:27:11.733847+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"51'192","reported_seq":207,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.729817+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:27:11.729817+0000","last_peered":"2024-09-22T03:27:11.729817+0000","last_clean":"2024-09-22T03:27:11.729817+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:27:11.729817+0000","last_undegraded":"2024-09-22T03:27:11.729817+0000","last_fullsized":"2024-09-22T03:27:11.729817+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"51'168","reported_seq":183,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.738313+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:27:11.738313+0000","last_peered":"2024-09-22T03:27:11.738313+0000","last_clean":"2024-09-22T03:27:11.738313+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:27:11.738313+0000","last_undegraded":"2024-09-22T03:27:11.738313+0000","last_fullsized":"2024-09-22T03:27:11.738313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"51'176","reported_seq":191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.734100+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:27:11.734100+0000","last_peered":"2024-09-22T03:27:11.734100+0000","last_clean":"2024-09-22T03:27:11.734100+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:27:11.734100+0000","last_undegraded":"2024-09-22T03:27:11.734100+0000","last_fullsized":"2024-09-22T03:27:11.734100+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"51'208","reported_seq":223,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735863+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:27:11.735863+0000","last_peered":"2024-09-22T03:27:11.735863+0000","last_clean":"2024-09-22T03:27:11.735863+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:27:11.735863+0000","last_undegraded":"2024-09-22T03:27:11.735863+0000","last_fullsized":"2024-09-22T03:27:11.735863+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"52'183","reported_seq":199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.802315+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:27:11.802315+0000","last_peered":"2024-09-22T03:27:11.802315+0000","last_clean":"2024-09-22T03:27:11.802315+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:27:11.802315+0000","last_undegraded":"2024-09-22T03:27:11.802315+0000","last_fullsized":"2024-09-22T03:27:11.802315+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"52'287","reported_seq":303,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.894548+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:27:11.894548+0000","last_peered":"2024-09-22T03:27:11.894548+0000","last_clean":"2024-09-22T03:27:11.894548+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:27:11.894548+0000","last_undegraded":"2024-09-22T03:27:11.894548+0000","last_fullsized":"2024-09-22T03:27:11.894548+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":259,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.727525+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:11.727525+0000","last_peered":"2024-09-22T03:27:11.727525+0000","last_clean":"2024-09-22T03:27:11.727525+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:11.727525+0000","last_undegraded":"2024-09-22T03:27:11.727525+0000","last_fullsized":"2024-09-22T03:27:11.727525+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":125,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.736450+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:11.736450+0000","last_peered":"2024-09-22T03:27:11.736450+0000","last_clean":"2024-09-22T03:27:11.736450+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:11.736450+0000","last_undegraded":"2024-09-22T03:27:11.736450+0000","last_fullsized":"2024-09-22T03:27:11.736450+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"51'216","reported_seq":231,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.740555+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:27:11.740555+0000","last_peered":"2024-09-22T03:27:11.740555+0000","last_clean":"2024-09-22T03:27:11.740555+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:27:11.740555+0000","last_undegraded":"2024-09-22T03:27:11.740555+0000","last_fullsized":"2024-09-22T03:27:11.740555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.740328+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:11.740328+0000","last_peered":"2024-09-22T03:27:11.740328+0000","last_clean":"2024-09-22T03:27:11.740328+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:11.740328+0000","last_undegraded":"2024-09-22T03:27:11.740328+0000","last_fullsized":"2024-09-22T03:27:11.740328+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"51'280","reported_seq":295,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.738189+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:27:11.738189+0000","last_peered":"2024-09-22T03:27:11.738189+0000","last_clean":"2024-09-22T03:27:11.738189+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:27:11.738189+0000","last_undegraded":"2024-09-22T03:27:11.738189+0000","last_fullsized":"2024-09-22T03:27:11.738189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.733568+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:11.733568+0000","last_peered":"2024-09-22T03:27:11.733568+0000","last_clean":"2024-09-22T03:27:11.733568+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:11.733568+0000","last_undegraded":"2024-09-22T03:27:11.733568+0000","last_fullsized":"2024-09-22T03:27:11.733568+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"51'328","reported_seq":343,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735150+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:27:11.735150+0000","last_peered":"2024-09-22T03:27:11.735150+0000","last_clean":"2024-09-22T03:27:11.735150+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:27:11.735150+0000","last_undegraded":"2024-09-22T03:27:11.735150+0000","last_fullsized":"2024-09-22T03:27:11.735150+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":328,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":126,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735858+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:11.735858+0000","last_peered":"2024-09-22T03:27:11.735858+0000","last_clean":"2024-09-22T03:27:11.735858+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:11.735858+0000","last_undegraded":"2024-09-22T03:27:11.735858+0000","last_fullsized":"2024-09-22T03:27:11.735858+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"51'216","reported_seq":231,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.732812+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:27:11.732812+0000","last_peered":"2024-09-22T03:27:11.732812+0000","last_clean":"2024-09-22T03:27:11.732812+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:27:11.732812+0000","last_undegraded":"2024-09-22T03:27:11.732812+0000","last_fullsized":"2024-09-22T03:27:11.732812+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.736987+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:11.736987+0000","last_peered":"2024-09-22T03:27:11.736987+0000","last_clean":"2024-09-22T03:27:11.736987+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:11.736987+0000","last_undegraded":"2024-09-22T03:27:11.736987+0000","last_fullsized":"2024-09-22T03:27:11.736987+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"52'295","reported_seq":311,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.947740+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:27:11.947740+0000","last_peered":"2024-09-22T03:27:11.947740+0000","last_clean":"2024-09-22T03:27:11.947740+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:27:11.947740+0000","last_undegraded":"2024-09-22T03:27:11.947740+0000","last_fullsized":"2024-09-22T03:27:11.947740+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.730826+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:11.730826+0000","last_peered":"2024-09-22T03:27:11.730826+0000","last_clean":"2024-09-22T03:27:11.730826+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:11.730826+0000","last_undegraded":"2024-09-22T03:27:11.730826+0000","last_fullsized":"2024-09-22T03:27:11.730826+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"51'248","reported_seq":263,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.729698+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:27:11.729698+0000","last_peered":"2024-09-22T03:27:11.729698+0000","last_clean":"2024-09-22T03:27:11.729698+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:27:11.729698+0000","last_undegraded":"2024-09-22T03:27:11.729698+0000","last_fullsized":"2024-09-22T03:27:11.729698+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.735914+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:11.735914+0000","last_peered":"2024-09-22T03:27:11.735914+0000","last_clean":"2024-09-22T03:27:11.735914+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:11.735914+0000","last_undegraded":"2024-09-22T03:27:11.735914+0000","last_fullsized":"2024-09-22T03:27:11.735914+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"51'248","reported_seq":263,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.739419+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:27:11.739419+0000","last_peered":"2024-09-22T03:27:11.739419+0000","last_clean":"2024-09-22T03:27:11.739419+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:27:11.739419+0000","last_undegraded":"2024-09-22T03:27:11.739419+0000","last_fullsized":"2024-09-22T03:27:11.739419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":124,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-22T03:27:11.736233+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:11.736233+0000","last_peered":"2024-09-22T03:27:11.736233+0000","last_clean":"2024-09-22T03:27:11.736233+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:11.736233+0000","last_undegraded":"2024-09-22T03:27:11.736233+0000","last_fullsized":"2024-09-22T03:27:11.736233+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":29622272,"num_objects":452,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214021099520,"internally_reserved":0,"allocated":190394368,"data_stored":190394368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3613,"ondisk_log_size":3613,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214021099520,"internally_reserved":0,"allocated":190394368,"data_stored":190394368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":42580,"kb_used_data":42580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52255148,"statfs":{"total":53552873472,"available":53509271552,"internally_reserved":0,"allocated":43601920,"data_stored":43601920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":54080,"kb_used_data":54080,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52243648,"statfs":{"total":53552873472,"available":53497495552,"internally_reserved":0,"allocated":55377920,"data_stored":55377920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":39612,"kb_used_data":39612,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52258116,"statfs":{"total":53552873472,"available":53512310784,"internally_reserved":0,"allocated":40562688,"data_stored":40562688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":46948,"kb_used_data":46948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52250780,"statfs":{"total":53552873472,"available":53504798720,"internally_reserved":0,"allocated":48074752,"data_stored":48074752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53502263296,"internally_reserved":0,"allocated":50610176,"data_stored":50610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53512196096,"internally_reserved":0,"allocated":40677376,"data_stored":40677376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53497487360,"internally_reserved":0,"allocated":55386112,"data_stored":55386112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53502263296,"internally_reserved":0,"allocated":50610176,"data_stored":50610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53512196096,"internally_reserved":0,"allocated":40677376,"data_stored":40677376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53497487360,"internally_reserved":0,"allocated":55386112,"data_stored":55386112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53509152768,"internally_reserved":0,"allocated":43720704,"data_stored":43720704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:15.645 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:27:15.645 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:15.645 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:27:15.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:27:15.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:15.799+0000 7ff127d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:15.808+0000 7ff127d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:15.843 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 4257 4241 1.38041 1.33594 0.174509 0.090338 2024-09-22T03:27:15.891 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:15.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:15.893 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:15.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:15.896 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:15.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:27:16.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.024+0000 7f285ccd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.034+0000 7f285ccd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.049 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:16.049 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:16.049 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:16.050 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:16.053 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:16.053 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:16.053 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:27:16.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.172+0000 7f30b9517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.185+0000 7f30b9517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.237 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:16.237 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:16.237 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:16.238 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:16.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:16.241 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:16.241 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:27:16.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.350+0000 7fe26ca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.359+0000 7fe26ca2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.442 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:16.442 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:16.442 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:16.442 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:16.442 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:16.442 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:16.442 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:16.443 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:16.443 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:16.443 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:16.443 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:16.443 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:16.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:16.445 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:16.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:16.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 4395 4379 1.36832 1.07812 0.0983919 0.091114 2024-09-22T03:27:16.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.855+0000 7f9fae4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:16.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:16.861+0000 7f9fae4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:17.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 4557 4541 1.36436 1.26562 0.099976 0.091443 2024-09-22T03:27:18.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 4716 4700 1.35984 1.24219 0.0634018 0.0917256 2024-09-22T03:27:19.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:19.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:19.154+0000 7f20eb22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:19.159+0000 7f20eb22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:19.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 4875 4859 1.35563 1.24219 0.0557881 0.0920197 2024-09-22T03:27:20.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 5065 5049 1.36006 1.48438 0.0559485 0.0917627 2024-09-22T03:27:21.353 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:21.445 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:21.446 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:21.446 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:21.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:21.472+0000 7f9fd1ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:21.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:21.478+0000 7f9fd1ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:21.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:21.564+0000 7ff8dd197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:21.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:21.572+0000 7ff8dd197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:21.817 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:27:21.818 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:27:21.832 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-22T03:27:19.982144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36782643,"num_objects":557,"num_object_clones":0,"num_object_copies":1671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4557,"num_write_kb":37240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4502,"ondisk_log_size":4502,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":224100,"kb_used_data":224100,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208966812,"statfs":{"total":214211493888,"available":213982015488,"internally_reserved":0,"allocated":229478400,"data_stored":229478400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6569984,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":802,"num_write_kb":6416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.264790"},"pg_stats":[{"pgid":"5.f","version":"55'207","reported_seq":232,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.863618+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:27:16.863618+0000","last_peered":"2024-09-22T03:27:16.863618+0000","last_clean":"2024-09-22T03:27:16.863618+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:27:16.863618+0000","last_undegraded":"2024-09-22T03:27:16.863618+0000","last_fullsized":"2024-09-22T03:27:16.863618+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"55'305","reported_seq":330,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.948618+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:27:16.948618+0000","last_peered":"2024-09-22T03:27:16.948618+0000","last_clean":"2024-09-22T03:27:16.948618+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:27:16.948618+0000","last_undegraded":"2024-09-22T03:27:16.948618+0000","last_fullsized":"2024-09-22T03:27:16.948618+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506752,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"55'255","reported_seq":280,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:15.080617+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:27:15.080617+0000","last_peered":"2024-09-22T03:27:15.080617+0000","last_clean":"2024-09-22T03:27:15.080617+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:27:15.080617+0000","last_undegraded":"2024-09-22T03:27:15.080617+0000","last_fullsized":"2024-09-22T03:27:15.080617+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"55'239","reported_seq":264,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.377425+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:27:16.377425+0000","last_peered":"2024-09-22T03:27:16.377425+0000","last_clean":"2024-09-22T03:27:16.377425+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:27:16.377425+0000","last_undegraded":"2024-09-22T03:27:16.377425+0000","last_fullsized":"2024-09-22T03:27:16.377425+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"55'231","reported_seq":256,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.727925+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:27:16.727925+0000","last_peered":"2024-09-22T03:27:16.727925+0000","last_clean":"2024-09-22T03:27:16.727925+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:27:16.727925+0000","last_undegraded":"2024-09-22T03:27:16.727925+0000","last_fullsized":"2024-09-22T03:27:16.727925+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"55'247","reported_seq":272,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.821017+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:27:16.821017+0000","last_peered":"2024-09-22T03:27:16.821017+0000","last_clean":"2024-09-22T03:27:16.821017+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:27:16.821017+0000","last_undegraded":"2024-09-22T03:27:16.821017+0000","last_fullsized":"2024-09-22T03:27:16.821017+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"55'239","reported_seq":264,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.948549+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:27:16.948549+0000","last_peered":"2024-09-22T03:27:16.948549+0000","last_clean":"2024-09-22T03:27:16.948549+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:27:16.948549+0000","last_undegraded":"2024-09-22T03:27:16.948549+0000","last_fullsized":"2024-09-22T03:27:16.948549+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"55'215","reported_seq":240,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:15.927909+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:27:15.927909+0000","last_peered":"2024-09-22T03:27:15.927909+0000","last_clean":"2024-09-22T03:27:15.927909+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:27:15.927909+0000","last_undegraded":"2024-09-22T03:27:15.927909+0000","last_fullsized":"2024-09-22T03:27:15.927909+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"55'311","reported_seq":336,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.310572+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:27:16.310572+0000","last_peered":"2024-09-22T03:27:16.310572+0000","last_clean":"2024-09-22T03:27:16.310572+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:27:16.310572+0000","last_undegraded":"2024-09-22T03:27:16.310572+0000","last_fullsized":"2024-09-22T03:27:16.310572+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.744310+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:14.744310+0000","last_peered":"2024-09-22T03:27:14.744310+0000","last_clean":"2024-09-22T03:27:14.744310+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:14.744310+0000","last_undegraded":"2024-09-22T03:27:14.744310+0000","last_fullsized":"2024-09-22T03:27:14.744310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":134,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.772027+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:14.772027+0000","last_peered":"2024-09-22T03:27:14.772027+0000","last_clean":"2024-09-22T03:27:14.772027+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:14.772027+0000","last_undegraded":"2024-09-22T03:27:14.772027+0000","last_fullsized":"2024-09-22T03:27:14.772027+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"55'247","reported_seq":272,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:15.282150+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:27:15.282150+0000","last_peered":"2024-09-22T03:27:15.282150+0000","last_clean":"2024-09-22T03:27:15.282150+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:27:15.282150+0000","last_undegraded":"2024-09-22T03:27:15.282150+0000","last_fullsized":"2024-09-22T03:27:15.282150+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.752756+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:14.752756+0000","last_peered":"2024-09-22T03:27:14.752756+0000","last_clean":"2024-09-22T03:27:14.752756+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:14.752756+0000","last_undegraded":"2024-09-22T03:27:14.752756+0000","last_fullsized":"2024-09-22T03:27:14.752756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"55'335","reported_seq":360,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:15.653077+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:27:15.653077+0000","last_peered":"2024-09-22T03:27:15.653077+0000","last_clean":"2024-09-22T03:27:15.653077+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:27:15.653077+0000","last_undegraded":"2024-09-22T03:27:15.653077+0000","last_fullsized":"2024-09-22T03:27:15.653077+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.768492+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.768492+0000","last_peered":"2024-09-22T03:27:14.768492+0000","last_clean":"2024-09-22T03:27:14.768492+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:14.768492+0000","last_undegraded":"2024-09-22T03:27:14.768492+0000","last_fullsized":"2024-09-22T03:27:14.768492+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"55'375","reported_seq":400,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.119235+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:27:16.119235+0000","last_peered":"2024-09-22T03:27:16.119235+0000","last_clean":"2024-09-22T03:27:16.119235+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:27:16.119235+0000","last_undegraded":"2024-09-22T03:27:16.119235+0000","last_fullsized":"2024-09-22T03:27:16.119235+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":135,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.750942+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.750942+0000","last_peered":"2024-09-22T03:27:14.750942+0000","last_clean":"2024-09-22T03:27:14.750942+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:14.750942+0000","last_undegraded":"2024-09-22T03:27:14.750942+0000","last_fullsized":"2024-09-22T03:27:14.750942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"55'263","reported_seq":288,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.912020+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:27:16.912020+0000","last_peered":"2024-09-22T03:27:16.912020+0000","last_clean":"2024-09-22T03:27:16.912020+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:27:16.912020+0000","last_undegraded":"2024-09-22T03:27:16.912020+0000","last_fullsized":"2024-09-22T03:27:16.912020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.756438+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.756438+0000","last_peered":"2024-09-22T03:27:14.756438+0000","last_clean":"2024-09-22T03:27:14.756438+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:14.756438+0000","last_undegraded":"2024-09-22T03:27:14.756438+0000","last_fullsized":"2024-09-22T03:27:14.756438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"55'335","reported_seq":360,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.170512+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:27:16.170512+0000","last_peered":"2024-09-22T03:27:16.170512+0000","last_clean":"2024-09-22T03:27:16.170512+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:27:16.170512+0000","last_undegraded":"2024-09-22T03:27:16.170512+0000","last_fullsized":"2024-09-22T03:27:16.170512+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.748396+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.748396+0000","last_peered":"2024-09-22T03:27:14.748396+0000","last_clean":"2024-09-22T03:27:14.748396+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:14.748396+0000","last_undegraded":"2024-09-22T03:27:14.748396+0000","last_fullsized":"2024-09-22T03:27:14.748396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"55'295","reported_seq":320,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:16.646033+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:27:16.646033+0000","last_peered":"2024-09-22T03:27:16.646033+0000","last_clean":"2024-09-22T03:27:16.646033+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:27:16.646033+0000","last_undegraded":"2024-09-22T03:27:16.646033+0000","last_fullsized":"2024-09-22T03:27:16.646033+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.753346+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:14.753346+0000","last_peered":"2024-09-22T03:27:14.753346+0000","last_clean":"2024-09-22T03:27:14.753346+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:14.753346+0000","last_undegraded":"2024-09-22T03:27:14.753346+0000","last_fullsized":"2024-09-22T03:27:14.753346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"55'303","reported_seq":328,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:15.709815+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:27:15.709815+0000","last_peered":"2024-09-22T03:27:15.709815+0000","last_clean":"2024-09-22T03:27:15.709815+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:27:15.709815+0000","last_undegraded":"2024-09-22T03:27:15.709815+0000","last_fullsized":"2024-09-22T03:27:15.709815+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.771801+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:14.771801+0000","last_peered":"2024-09-22T03:27:14.771801+0000","last_clean":"2024-09-22T03:27:14.771801+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:14.771801+0000","last_undegraded":"2024-09-22T03:27:14.771801+0000","last_fullsized":"2024-09-22T03:27:14.771801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":36192256,"num_objects":553,"num_object_clones":0,"num_object_copies":1659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4418,"num_write_kb":35344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213979762688,"internally_reserved":0,"allocated":231731200,"data_stored":231731200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4402,"ondisk_log_size":4402,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213979770880,"internally_reserved":0,"allocated":231723008,"data_stored":231723008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53498437632,"internally_reserved":0,"allocated":54435840,"data_stored":54435840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":53144,"kb_used_data":53144,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52244584,"statfs":{"total":53552873472,"available":53498454016,"internally_reserved":0,"allocated":54419456,"data_stored":54419456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":65708,"kb_used_data":65708,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52232020,"statfs":{"total":53552873472,"available":53485588480,"internally_reserved":0,"allocated":67284992,"data_stored":67284992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":46800,"kb_used_data":46800,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52250928,"statfs":{"total":53552873472,"available":53504950272,"internally_reserved":0,"allocated":47923200,"data_stored":47923200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":58448,"kb_used_data":58448,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52239280,"statfs":{"total":53552873472,"available":53493022720,"internally_reserved":0,"allocated":59850752,"data_stored":59850752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53498437632,"internally_reserved":0,"allocated":54435840,"data_stored":54435840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53491011584,"internally_reserved":0,"allocated":61861888,"data_stored":61861888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53504876544,"internally_reserved":0,"allocated":47996928,"data_stored":47996928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53485445120,"internally_reserved":0,"allocated":67428352,"data_stored":67428352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53498437632,"internally_reserved":0,"allocated":54435840,"data_stored":54435840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53491011584,"internally_reserved":0,"allocated":61861888,"data_stored":61861888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53504868352,"internally_reserved":0,"allocated":48005120,"data_stored":48005120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53485445120,"internally_reserved":0,"allocated":67428352,"data_stored":67428352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53498437632,"internally_reserved":0,"allocated":54435840,"data_stored":54435840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:21.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 5206 5190 1.35144 1.10156 0.202227 0.0921741 2024-09-22T03:27:22.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 5383 5367 1.35245 1.38281 0.109053 0.0922453 2024-09-22T03:27:23.593 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:23.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:23.735+0000 7f6fc7f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:23.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:23.743+0000 7f6fc7f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:23.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 16 5572 5556 1.35633 1.47656 0.122083 0.0920472 2024-09-22T03:27:24.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 5749 5733 1.35713 1.38281 0.0446297 0.0919531 2024-09-22T03:27:25.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 5885 5869 1.34846 1.0625 0.0838874 0.0925677 2024-09-22T03:27:26.006 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:26.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:26.122+0000 7f0f4ad4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:26.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:26.127+0000 7f0f4ad4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:26.834 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:26.834 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:26.834 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:27:26.835 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:27:26.844 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 6062 6046 1.34944 1.38281 0.143065 0.0924606 2024-09-22T03:27:26.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:26.950+0000 7f0d1e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:26.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:26.963+0000 7f0d1e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:27.068 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:27.069 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:27.072 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:27.072 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:27.073 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:27:27.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:27.260+0000 7f15b186c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:27.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:27.275+0000 7f15b186c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:27.312 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:27.313 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:27.313 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:27.313 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:27.316 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:27.316 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:27.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:27:27.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:27.479+0000 7fd9c6f4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:27.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:27.491+0000 7fd9c6f4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:27.535 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:27.536 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:27.540 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:27.540 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:27.540 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:27:27.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:27.658+0000 7f97f54b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:27.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:27.668+0000 7f97f54b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:27.712 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:27.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:27.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:27.713 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:27.713 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:27.715 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:27.716 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:27.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 6231 6215 1.34862 1.32031 0.117528 0.0924987 2024-09-22T03:27:28.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:28.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:28.367+0000 7fd7ddafa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:28.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:28.373+0000 7fd7ddafa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:28.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 6406 6390 1.34912 1.36719 0.12958 0.0925282 2024-09-22T03:27:29.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 6569 6553 1.34713 1.27344 0.0541596 0.0926482 2024-09-22T03:27:30.545 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:30.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:30.652+0000 7fae6a757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:30.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:30.657+0000 7fae6a757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:30.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 6736 6720 1.34604 1.30469 0.148364 0.0927487 2024-09-22T03:27:31.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:27:31.844805+0000 min lat: 0.0117735 max lat: 0.474035 avg lat: 0.0923836 2024-09-22T03:27:31.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:27:31.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 6934 6918 1.35105 1.54688 0.0238564 0.0923836 2024-09-22T03:27:32.717 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:32.717 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:32.717 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:27:32.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:27:32.829 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:32.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 7099 7083 1.34954 1.28906 0.136141 0.0923939 2024-09-22T03:27:32.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:32.837+0000 7f1c2b917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:32.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:32.849+0000 7f1c2b917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:32.892 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:32.892 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:32.892 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:32.892 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:32.892 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:32.892 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:32.892 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:32.893 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:32.893 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:32.893 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:32.893 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:32.893 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:32.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:32.896 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:32.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:27:32.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:32.948+0000 7f8daa6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:32.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:32.954+0000 7f8daa6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:33.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:33.051+0000 7fe9e5797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:33.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:33.060+0000 7fe9e5797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:33.076 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:33.077 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:33.080 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:33.080 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:33.080 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:27:33.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:33.233+0000 7f3e8b6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:33.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:33.242+0000 7f3e8b6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:33.260 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:33.260 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:33.260 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:33.260 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:33.261 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:33.263 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:33.263 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:33.263 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:27:33.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:33.418+0000 7f830ab17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:33.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:33.428+0000 7f830ab17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:33.453 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:33.454 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:33.454 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:33.454 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:33.454 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:33.456 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:33.457 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:33.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 7258 7242 1.34698 1.24219 0.130426 0.0926169 2024-09-22T03:27:34.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 16 7398 7382 1.34109 1.09375 0.194922 0.0930106 2024-09-22T03:27:35.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:35.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:35.406+0000 7f11f4e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:35.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:35.412+0000 7f11f4e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:35.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 7569 7553 1.34097 1.33594 0.112421 0.0930517 2024-09-22T03:27:36.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 7716 7700 1.33669 1.14844 0.120739 0.0932794 2024-09-22T03:27:37.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:37.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:37.770+0000 7f4f2f46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:37.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:37.776+0000 7f4f2f46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:37.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 7879 7863 1.33531 1.27344 0.0591117 0.0935044 2024-09-22T03:27:38.457 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:38.457 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:38.458 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:27:38.458 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:27:38.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:38.587+0000 7fa00a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:38.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:38.598+0000 7fa00a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:38.663 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:38.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:38.666 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:38.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:27:38.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:38.783+0000 7f1cc5eb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:38.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:38.794+0000 7f1cc5eb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:38.809 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:38.810 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:38.810 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:38.810 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:38.810 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:38.810 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:38.812 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:38.812 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:38.812 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:27:38.845 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 16 8055 8039 1.33615 1.375 0.175548 0.0933542 2024-09-22T03:27:38.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:38.930+0000 7f1ec910e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:38.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:38.938+0000 7f1ec910e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:38.957 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:38.958 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:38.959 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:38.961 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:38.961 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:38.961 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:27:39.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:39.132+0000 7f95de957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:39.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:39.144+0000 7f95de957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:39.220 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:39.220 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:39.220 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:39.220 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:39.220 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:39.221 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:39.221 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:39.221 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:39.221 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:39.221 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:39.221 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:39.221 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:39.224 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:39.224 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:39.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 8202 8186 1.33224 1.14844 0.0885852 0.0936692 2024-09-22T03:27:39.942 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:40.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:40.083+0000 7f95e9cba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:40.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:40.091+0000 7f95e9cba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:40.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 8361 8345 1.3304 1.24219 0.0579658 0.0937574 2024-09-22T03:27:41.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 8541 8525 1.33192 1.40625 0.0305008 0.0937428 2024-09-22T03:27:42.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:42.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:42.428+0000 7f61712b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:42.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:42.434+0000 7f61712b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:42.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 8710 8694 1.33169 1.32031 0.128367 0.0937377 2024-09-22T03:27:43.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 8867 8851 1.32966 1.22656 0.0590855 0.093823 2024-09-22T03:27:44.225 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:44.225 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:44.226 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:27:44.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:27:44.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.338+0000 7ff859ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.347+0000 7ff859ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.392 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:44.392 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:44.392 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:44.392 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:44.393 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:44.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:44.396 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:44.396 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:27:44.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.505+0000 7f9d07a23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.515+0000 7f9d07a23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.550 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:44.550 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:44.550 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:44.550 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:44.550 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:44.550 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:44.550 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:44.551 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:44.551 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:44.551 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:44.551 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:44.551 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:44.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:44.554 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:44.554 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:27:44.689 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:44.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.682+0000 7f49a4197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.692+0000 7f49a4197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:44.741 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:44.742 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:44.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:27:44.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.810+0000 7fb087223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.815+0000 7fb087223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 9018 9002 1.32683 1.17969 0.0881694 0.0941189 2024-09-22T03:27:44.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.917+0000 7fafb9f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:44.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:44.928+0000 7fafb9f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:27:45.036 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:27:45.040 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:27:45.040 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:27:45.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 9154 9138 1.32193 1.0625 0.10685 0.0944747 2024-09-22T03:27:46.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 9324 9308 1.32204 1.32812 0.0765608 0.0944464 2024-09-22T03:27:47.044 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:47.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:47.174+0000 7f94f8b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:47.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:47.180+0000 7f94f8b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:47.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 9478 9462 1.31992 1.20312 0.162402 0.0945835 2024-09-22T03:27:48.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 9628 9612 1.31732 1.17188 0.134044 0.0947969 2024-09-22T03:27:49.375 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:49.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:49.505+0000 7f04a202c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:49.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:49.510+0000 7f04a202c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:49.846 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 9792 9776 1.3167 1.28125 0.179297 0.0948133 2024-09-22T03:27:50.041 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:50.041 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:50.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:50.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:50.186+0000 7fe8170d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:50.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:50.195+0000 7fe8170d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:50.396 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:27:50.397 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:27:50.415 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-22T03:27:49.987697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77054515,"num_objects":1172,"num_object_clones":0,"num_object_copies":3516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9473,"num_write_kb":76568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9418,"ondisk_log_size":9418,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":468008,"kb_used_data":468008,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208722904,"statfs":{"total":214211493888,"available":213732253696,"internally_reserved":0,"allocated":479240192,"data_stored":479240192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13156352,"num_objects":202,"num_object_clones":0,"num_object_copies":606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1606,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002287"},"pg_stats":[{"pgid":"5.f","version":"55'551","reported_seq":576,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:45.160641+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:27:45.160641+0000","last_peered":"2024-09-22T03:27:45.160641+0000","last_clean":"2024-09-22T03:27:45.160641+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:27:45.160641+0000","last_undegraded":"2024-09-22T03:27:45.160641+0000","last_fullsized":"2024-09-22T03:27:45.160641+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"55'655","reported_seq":680,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:45.871055+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:27:45.871055+0000","last_peered":"2024-09-22T03:27:45.871055+0000","last_clean":"2024-09-22T03:27:45.871055+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:27:45.871055+0000","last_undegraded":"2024-09-22T03:27:45.871055+0000","last_fullsized":"2024-09-22T03:27:45.871055+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"55'655","reported_seq":680,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:44.598895+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:27:44.598895+0000","last_peered":"2024-09-22T03:27:44.598895+0000","last_clean":"2024-09-22T03:27:44.598895+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:27:44.598895+0000","last_undegraded":"2024-09-22T03:27:44.598895+0000","last_fullsized":"2024-09-22T03:27:44.598895+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"55'583","reported_seq":608,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.294798+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:27:46.294798+0000","last_peered":"2024-09-22T03:27:46.294798+0000","last_clean":"2024-09-22T03:27:46.294798+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:27:46.294798+0000","last_undegraded":"2024-09-22T03:27:46.294798+0000","last_fullsized":"2024-09-22T03:27:46.294798+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"55'521","reported_seq":546,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.900821+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:27:46.900821+0000","last_peered":"2024-09-22T03:27:46.900821+0000","last_clean":"2024-09-22T03:27:46.900821+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:27:46.900821+0000","last_undegraded":"2024-09-22T03:27:46.900821+0000","last_fullsized":"2024-09-22T03:27:46.900821+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4276224,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":522,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"55'523","reported_seq":548,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.952842+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:27:46.952842+0000","last_peered":"2024-09-22T03:27:46.952842+0000","last_clean":"2024-09-22T03:27:46.952842+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:27:46.952842+0000","last_undegraded":"2024-09-22T03:27:46.952842+0000","last_fullsized":"2024-09-22T03:27:46.952842+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4292608,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":524,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"55'567","reported_seq":592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.496338+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:27:46.496338+0000","last_peered":"2024-09-22T03:27:46.496338+0000","last_clean":"2024-09-22T03:27:46.496338+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:27:46.496338+0000","last_undegraded":"2024-09-22T03:27:46.496338+0000","last_fullsized":"2024-09-22T03:27:46.496338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"55'463","reported_seq":488,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:42.606172+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:27:42.606172+0000","last_peered":"2024-09-22T03:27:42.606172+0000","last_clean":"2024-09-22T03:27:42.606172+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:27:42.606172+0000","last_undegraded":"2024-09-22T03:27:42.606172+0000","last_fullsized":"2024-09-22T03:27:42.606172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"55'647","reported_seq":672,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.670503+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:27:46.670503+0000","last_peered":"2024-09-22T03:27:46.670503+0000","last_clean":"2024-09-22T03:27:46.670503+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:27:46.670503+0000","last_undegraded":"2024-09-22T03:27:46.670503+0000","last_fullsized":"2024-09-22T03:27:46.670503+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.744310+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:14.744310+0000","last_peered":"2024-09-22T03:27:14.744310+0000","last_clean":"2024-09-22T03:27:14.744310+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:14.744310+0000","last_undegraded":"2024-09-22T03:27:14.744310+0000","last_fullsized":"2024-09-22T03:27:14.744310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":134,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.772027+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:14.772027+0000","last_peered":"2024-09-22T03:27:14.772027+0000","last_clean":"2024-09-22T03:27:14.772027+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:14.772027+0000","last_undegraded":"2024-09-22T03:27:14.772027+0000","last_fullsized":"2024-09-22T03:27:14.772027+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"55'551","reported_seq":576,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.887437+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:27:46.887437+0000","last_peered":"2024-09-22T03:27:46.887437+0000","last_clean":"2024-09-22T03:27:46.887437+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:27:46.887437+0000","last_undegraded":"2024-09-22T03:27:46.887437+0000","last_fullsized":"2024-09-22T03:27:46.887437+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.752756+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:14.752756+0000","last_peered":"2024-09-22T03:27:14.752756+0000","last_clean":"2024-09-22T03:27:14.752756+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:14.752756+0000","last_undegraded":"2024-09-22T03:27:14.752756+0000","last_fullsized":"2024-09-22T03:27:14.752756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"55'575","reported_seq":600,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:45.953876+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:27:45.953876+0000","last_peered":"2024-09-22T03:27:45.953876+0000","last_clean":"2024-09-22T03:27:45.953876+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:27:45.953876+0000","last_undegraded":"2024-09-22T03:27:45.953876+0000","last_fullsized":"2024-09-22T03:27:45.953876+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.768492+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.768492+0000","last_peered":"2024-09-22T03:27:14.768492+0000","last_clean":"2024-09-22T03:27:14.768492+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:14.768492+0000","last_undegraded":"2024-09-22T03:27:14.768492+0000","last_fullsized":"2024-09-22T03:27:14.768492+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"55'719","reported_seq":744,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.442668+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:27:46.442668+0000","last_peered":"2024-09-22T03:27:46.442668+0000","last_clean":"2024-09-22T03:27:46.442668+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:27:46.442668+0000","last_undegraded":"2024-09-22T03:27:46.442668+0000","last_fullsized":"2024-09-22T03:27:46.442668+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":135,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.750942+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.750942+0000","last_peered":"2024-09-22T03:27:14.750942+0000","last_clean":"2024-09-22T03:27:14.750942+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:14.750942+0000","last_undegraded":"2024-09-22T03:27:14.750942+0000","last_fullsized":"2024-09-22T03:27:14.750942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"55'543","reported_seq":568,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.182578+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:27:46.182578+0000","last_peered":"2024-09-22T03:27:46.182578+0000","last_clean":"2024-09-22T03:27:46.182578+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:27:46.182578+0000","last_undegraded":"2024-09-22T03:27:46.182578+0000","last_fullsized":"2024-09-22T03:27:46.182578+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.756438+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.756438+0000","last_peered":"2024-09-22T03:27:14.756438+0000","last_clean":"2024-09-22T03:27:14.756438+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:14.756438+0000","last_undegraded":"2024-09-22T03:27:14.756438+0000","last_fullsized":"2024-09-22T03:27:14.756438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"55'583","reported_seq":608,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:44.716326+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:27:44.716326+0000","last_peered":"2024-09-22T03:27:44.716326+0000","last_clean":"2024-09-22T03:27:44.716326+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:27:44.716326+0000","last_undegraded":"2024-09-22T03:27:44.716326+0000","last_fullsized":"2024-09-22T03:27:44.716326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.748396+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.748396+0000","last_peered":"2024-09-22T03:27:14.748396+0000","last_clean":"2024-09-22T03:27:14.748396+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:14.748396+0000","last_undegraded":"2024-09-22T03:27:14.748396+0000","last_fullsized":"2024-09-22T03:27:14.748396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"55'567","reported_seq":592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.565940+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:27:46.565940+0000","last_peered":"2024-09-22T03:27:46.565940+0000","last_clean":"2024-09-22T03:27:46.565940+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:27:46.565940+0000","last_undegraded":"2024-09-22T03:27:46.565940+0000","last_fullsized":"2024-09-22T03:27:46.565940+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.753346+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:14.753346+0000","last_peered":"2024-09-22T03:27:14.753346+0000","last_clean":"2024-09-22T03:27:14.753346+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:14.753346+0000","last_undegraded":"2024-09-22T03:27:14.753346+0000","last_fullsized":"2024-09-22T03:27:14.753346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"55'615","reported_seq":640,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:46.826185+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:27:46.826185+0000","last_peered":"2024-09-22T03:27:46.826185+0000","last_clean":"2024-09-22T03:27:46.826185+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:27:46.826185+0000","last_undegraded":"2024-09-22T03:27:46.826185+0000","last_fullsized":"2024-09-22T03:27:46.826185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.771801+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:14.771801+0000","last_peered":"2024-09-22T03:27:14.771801+0000","last_clean":"2024-09-22T03:27:14.771801+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:14.771801+0000","last_undegraded":"2024-09-22T03:27:14.771801+0000","last_fullsized":"2024-09-22T03:27:14.771801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":76464128,"num_objects":1168,"num_object_clones":0,"num_object_copies":3504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9334,"num_write_kb":74672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213727531008,"internally_reserved":0,"allocated":483962880,"data_stored":483962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9318,"ondisk_log_size":9318,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213727531008,"internally_reserved":0,"allocated":483962880,"data_stored":483962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53440495616,"internally_reserved":0,"allocated":112377856,"data_stored":112377856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":109664,"kb_used_data":109664,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52188064,"statfs":{"total":53552873472,"available":53440577536,"internally_reserved":0,"allocated":112295936,"data_stored":112295936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":136340,"kb_used_data":136340,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52161388,"statfs":{"total":53552873472,"available":53413261312,"internally_reserved":0,"allocated":139612160,"data_stored":139612160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":97648,"kb_used_data":97648,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52200080,"statfs":{"total":53552873472,"available":53452881920,"internally_reserved":0,"allocated":99991552,"data_stored":99991552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":124356,"kb_used_data":124356,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52173372,"statfs":{"total":53552873472,"available":53425532928,"internally_reserved":0,"allocated":127340544,"data_stored":127340544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53440495616,"internally_reserved":0,"allocated":112377856,"data_stored":112377856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53423431680,"internally_reserved":0,"allocated":129441792,"data_stored":129441792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53452881920,"internally_reserved":0,"allocated":99991552,"data_stored":99991552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53410721792,"internally_reserved":0,"allocated":142151680,"data_stored":142151680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53440495616,"internally_reserved":0,"allocated":112377856,"data_stored":112377856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53423431680,"internally_reserved":0,"allocated":129441792,"data_stored":129441792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53452881920,"internally_reserved":0,"allocated":99991552,"data_stored":99991552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53410721792,"internally_reserved":0,"allocated":142151680,"data_stored":142151680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53440495616,"internally_reserved":0,"allocated":112377856,"data_stored":112377856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:50.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 9961 9945 1.31676 1.32031 0.0778502 0.0948724 2024-09-22T03:27:51.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:51.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:27:51.846559+0000 min lat: 0.0117735 max lat: 0.474035 avg lat: 0.094708 2024-09-22T03:27:51.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:27:51.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 10141 10125 1.31824 1.40625 0.095824 0.094708 2024-09-22T03:27:51.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:51.958+0000 7f49cd218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:51.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:51.964+0000 7f49cd218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:52.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 10303 10287 1.31738 1.26562 0.0589577 0.0948201 2024-09-22T03:27:53.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 10447 10431 1.31428 1.125 0.196085 0.0949808 2024-09-22T03:27:54.147 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:54.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:54.264+0000 7f51a230e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:54.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:54.271+0000 7f51a230e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:54.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 10598 10582 1.31214 1.17969 0.0588494 0.0951826 2024-09-22T03:27:55.417 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:27:55.417 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:27:55.418 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:27:55.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:55.536+0000 7f00db397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:55.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:55.543+0000 7f00db397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:55.746 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:27:55.747 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:27:55.761 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-22T03:27:53.988509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":83698227,"num_objects":1273,"num_object_clones":0,"num_object_copies":3819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10284,"num_write_kb":83056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10229,"ondisk_log_size":10229,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":506968,"kb_used_data":506968,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208683944,"statfs":{"total":214211493888,"available":213692358656,"internally_reserved":0,"allocated":519135232,"data_stored":519135232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13033472,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1591,"num_write_kb":12728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002356"},"pg_stats":[{"pgid":"5.f","version":"55'591","reported_seq":616,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.692448+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:27:51.692448+0000","last_peered":"2024-09-22T03:27:51.692448+0000","last_clean":"2024-09-22T03:27:51.692448+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:27:51.692448+0000","last_undegraded":"2024-09-22T03:27:51.692448+0000","last_fullsized":"2024-09-22T03:27:51.692448+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"55'751","reported_seq":776,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.716511+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:27:51.716511+0000","last_peered":"2024-09-22T03:27:51.716511+0000","last_clean":"2024-09-22T03:27:51.716511+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:27:51.716511+0000","last_undegraded":"2024-09-22T03:27:51.716511+0000","last_fullsized":"2024-09-22T03:27:51.716511+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"55'687","reported_seq":712,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.653014+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:27:51.653014+0000","last_peered":"2024-09-22T03:27:51.653014+0000","last_clean":"2024-09-22T03:27:51.653014+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:27:51.653014+0000","last_undegraded":"2024-09-22T03:27:51.653014+0000","last_fullsized":"2024-09-22T03:27:51.653014+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"55'639","reported_seq":664,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:50.769653+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:27:50.769653+0000","last_peered":"2024-09-22T03:27:50.769653+0000","last_clean":"2024-09-22T03:27:50.769653+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:27:50.769653+0000","last_undegraded":"2024-09-22T03:27:50.769653+0000","last_fullsized":"2024-09-22T03:27:50.769653+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"55'584","reported_seq":609,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.879884+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:27:51.879884+0000","last_peered":"2024-09-22T03:27:51.879884+0000","last_clean":"2024-09-22T03:27:51.879884+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:27:51.879884+0000","last_undegraded":"2024-09-22T03:27:51.879884+0000","last_fullsized":"2024-09-22T03:27:51.879884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4792320,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":585,"num_write_kb":4680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"55'583","reported_seq":608,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.367227+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:27:51.367227+0000","last_peered":"2024-09-22T03:27:51.367227+0000","last_clean":"2024-09-22T03:27:51.367227+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:27:51.367227+0000","last_undegraded":"2024-09-22T03:27:51.367227+0000","last_fullsized":"2024-09-22T03:27:51.367227+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"55'591","reported_seq":616,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:49.928122+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:27:49.928122+0000","last_peered":"2024-09-22T03:27:49.928122+0000","last_clean":"2024-09-22T03:27:49.928122+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:27:49.928122+0000","last_undegraded":"2024-09-22T03:27:49.928122+0000","last_fullsized":"2024-09-22T03:27:49.928122+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"55'535","reported_seq":560,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.149672+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:27:51.149672+0000","last_peered":"2024-09-22T03:27:51.149672+0000","last_clean":"2024-09-22T03:27:51.149672+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:27:51.149672+0000","last_undegraded":"2024-09-22T03:27:51.149672+0000","last_fullsized":"2024-09-22T03:27:51.149672+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"55'695","reported_seq":720,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.867108+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:27:51.867108+0000","last_peered":"2024-09-22T03:27:51.867108+0000","last_clean":"2024-09-22T03:27:51.867108+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:27:51.867108+0000","last_undegraded":"2024-09-22T03:27:51.867108+0000","last_fullsized":"2024-09-22T03:27:51.867108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.744310+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:14.744310+0000","last_peered":"2024-09-22T03:27:14.744310+0000","last_clean":"2024-09-22T03:27:14.744310+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:14.744310+0000","last_undegraded":"2024-09-22T03:27:14.744310+0000","last_fullsized":"2024-09-22T03:27:14.744310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":134,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.772027+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:14.772027+0000","last_peered":"2024-09-22T03:27:14.772027+0000","last_clean":"2024-09-22T03:27:14.772027+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:14.772027+0000","last_undegraded":"2024-09-22T03:27:14.772027+0000","last_fullsized":"2024-09-22T03:27:14.772027+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"55'615","reported_seq":640,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.177576+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:27:51.177576+0000","last_peered":"2024-09-22T03:27:51.177576+0000","last_clean":"2024-09-22T03:27:51.177576+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:27:51.177576+0000","last_undegraded":"2024-09-22T03:27:51.177576+0000","last_fullsized":"2024-09-22T03:27:51.177576+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.752756+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:14.752756+0000","last_peered":"2024-09-22T03:27:14.752756+0000","last_clean":"2024-09-22T03:27:14.752756+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:14.752756+0000","last_undegraded":"2024-09-22T03:27:14.752756+0000","last_fullsized":"2024-09-22T03:27:14.752756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"55'623","reported_seq":648,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:49.639914+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:27:49.639914+0000","last_peered":"2024-09-22T03:27:49.639914+0000","last_clean":"2024-09-22T03:27:49.639914+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:27:49.639914+0000","last_undegraded":"2024-09-22T03:27:49.639914+0000","last_fullsized":"2024-09-22T03:27:49.639914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.768492+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.768492+0000","last_peered":"2024-09-22T03:27:14.768492+0000","last_clean":"2024-09-22T03:27:14.768492+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:14.768492+0000","last_undegraded":"2024-09-22T03:27:14.768492+0000","last_fullsized":"2024-09-22T03:27:14.768492+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"55'767","reported_seq":792,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.444330+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:27:51.444330+0000","last_peered":"2024-09-22T03:27:51.444330+0000","last_clean":"2024-09-22T03:27:51.444330+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:27:51.444330+0000","last_undegraded":"2024-09-22T03:27:51.444330+0000","last_fullsized":"2024-09-22T03:27:51.444330+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":135,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.750942+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.750942+0000","last_peered":"2024-09-22T03:27:14.750942+0000","last_clean":"2024-09-22T03:27:14.750942+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:14.750942+0000","last_undegraded":"2024-09-22T03:27:14.750942+0000","last_fullsized":"2024-09-22T03:27:14.750942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"55'599","reported_seq":624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:50.653098+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:27:50.653098+0000","last_peered":"2024-09-22T03:27:50.653098+0000","last_clean":"2024-09-22T03:27:50.653098+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:27:50.653098+0000","last_undegraded":"2024-09-22T03:27:50.653098+0000","last_fullsized":"2024-09-22T03:27:50.653098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.756438+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.756438+0000","last_peered":"2024-09-22T03:27:14.756438+0000","last_clean":"2024-09-22T03:27:14.756438+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:14.756438+0000","last_undegraded":"2024-09-22T03:27:14.756438+0000","last_fullsized":"2024-09-22T03:27:14.756438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"55'631","reported_seq":656,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.754985+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:27:51.754985+0000","last_peered":"2024-09-22T03:27:51.754985+0000","last_clean":"2024-09-22T03:27:51.754985+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:27:51.754985+0000","last_undegraded":"2024-09-22T03:27:51.754985+0000","last_fullsized":"2024-09-22T03:27:51.754985+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.748396+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.748396+0000","last_peered":"2024-09-22T03:27:14.748396+0000","last_clean":"2024-09-22T03:27:14.748396+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:14.748396+0000","last_undegraded":"2024-09-22T03:27:14.748396+0000","last_fullsized":"2024-09-22T03:27:14.748396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"55'591","reported_seq":616,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.205218+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:27:51.205218+0000","last_peered":"2024-09-22T03:27:51.205218+0000","last_clean":"2024-09-22T03:27:51.205218+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:27:51.205218+0000","last_undegraded":"2024-09-22T03:27:51.205218+0000","last_fullsized":"2024-09-22T03:27:51.205218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.753346+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:14.753346+0000","last_peered":"2024-09-22T03:27:14.753346+0000","last_clean":"2024-09-22T03:27:14.753346+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:14.753346+0000","last_undegraded":"2024-09-22T03:27:14.753346+0000","last_fullsized":"2024-09-22T03:27:14.753346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"55'647","reported_seq":672,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:51.517863+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:27:51.517863+0000","last_peered":"2024-09-22T03:27:51.517863+0000","last_clean":"2024-09-22T03:27:51.517863+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:27:51.517863+0000","last_undegraded":"2024-09-22T03:27:51.517863+0000","last_fullsized":"2024-09-22T03:27:51.517863+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.771801+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:14.771801+0000","last_peered":"2024-09-22T03:27:14.771801+0000","last_clean":"2024-09-22T03:27:14.771801+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:14.771801+0000","last_undegraded":"2024-09-22T03:27:14.771801+0000","last_fullsized":"2024-09-22T03:27:14.771801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":83107840,"num_objects":1269,"num_object_clones":0,"num_object_copies":3807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10145,"num_write_kb":81160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213685952512,"internally_reserved":0,"allocated":525541376,"data_stored":525541376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10129,"ondisk_log_size":10129,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213685952512,"internally_reserved":0,"allocated":525541376,"data_stored":525541376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53430648832,"internally_reserved":0,"allocated":122224640,"data_stored":122224640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":119360,"kb_used_data":119360,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52178368,"statfs":{"total":53552873472,"available":53430648832,"internally_reserved":0,"allocated":122224640,"data_stored":122224640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":148460,"kb_used_data":148460,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52149268,"statfs":{"total":53552873472,"available":53400850432,"internally_reserved":0,"allocated":152023040,"data_stored":152023040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":104480,"kb_used_data":104480,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52193248,"statfs":{"total":53552873472,"available":53445885952,"internally_reserved":0,"allocated":106987520,"data_stored":106987520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":134668,"kb_used_data":134668,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163060,"statfs":{"total":53552873472,"available":53414973440,"internally_reserved":0,"allocated":137900032,"data_stored":137900032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53430648832,"internally_reserved":0,"allocated":122224640,"data_stored":122224640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53412372480,"internally_reserved":0,"allocated":140500992,"data_stored":140500992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53444161536,"internally_reserved":0,"allocated":108711936,"data_stored":108711936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53398769664,"internally_reserved":0,"allocated":154103808,"data_stored":154103808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53430648832,"internally_reserved":0,"allocated":122224640,"data_stored":122224640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53412372480,"internally_reserved":0,"allocated":140500992,"data_stored":140500992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53444161536,"internally_reserved":0,"allocated":108711936,"data_stored":108711936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53398769664,"internally_reserved":0,"allocated":154103808,"data_stored":154103808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53430648832,"internally_reserved":0,"allocated":122224640,"data_stored":122224640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:27:55.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 16 10756 10740 1.31092 1.23438 0.106327 0.0952758 2024-09-22T03:27:56.550 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:27:56.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:56.673+0000 7f7a0dd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:56.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:56.679+0000 7f7a0dd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:56.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 10902 10886 1.3083 1.14062 0.150892 0.0953543 2024-09-22T03:27:57.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 11073 11057 1.30872 1.33594 0.0632412 0.0953861 2024-09-22T03:27:58.784 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:27:58.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 11223 11207 1.30667 1.17188 0.0688258 0.0955608 2024-09-22T03:27:58.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:58.903+0000 7f57afd0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:58.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:27:58.908+0000 7f57afd0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:27:59.847 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 11372 11356 1.30457 1.16406 0.0471146 0.0957139 2024-09-22T03:28:00.763 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:00.764 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:00.764 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:28:00.764 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:28:00.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 11534 11518 1.30401 1.26562 0.177152 0.0957186 2024-09-22T03:28:00.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:00.880+0000 7f23d5863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:00.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:00.889+0000 7f23d5863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:00.955 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:00.957 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:00.958 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:00.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:28:01.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.075+0000 7efdf8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.085+0000 7efdf8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.097 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:01.128 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:01.128 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:01.128 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:01.128 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:01.128 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:01.128 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:01.128 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:01.129 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:01.129 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:01.129 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:01.129 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:01.129 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:01.131 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:01.131 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:01.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:28:01.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.235+0000 7f9020557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.242+0000 7f9020557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.304+0000 7fc924f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.316+0000 7fc924f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:01.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:01.346 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:01.346 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:28:01.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.465+0000 7fd922863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:01.476+0000 7fd922863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:01.496 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:01.496 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:01.496 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:01.497 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:01.500 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:01.500 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:01.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 11679 11663 1.30156 1.13281 0.0721379 0.0959109 2024-09-22T03:28:02.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 11835 11819 1.30039 1.21875 0.0760135 0.095973 2024-09-22T03:28:03.539 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:03.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:03.666+0000 7fd24fa2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:03.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:03.673+0000 7fd24fa2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:03.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 12000 11984 1.30023 1.28906 0.169667 0.0959843 2024-09-22T03:28:04.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 12160 12144 1.29954 1.25 0.0927301 0.0961156 2024-09-22T03:28:05.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:05.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 12325 12309 1.2994 1.28906 0.026149 0.0960927 2024-09-22T03:28:05.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:05.973+0000 7f13f8b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:05.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:05.979+0000 7f13f8b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:06.502 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:06.502 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:06.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:06.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:06.636+0000 7f7851f4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:06.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:06.647+0000 7f7851f4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:06.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 12482 12466 1.29843 1.22656 0.112337 0.0961741 2024-09-22T03:28:06.979 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:06.980 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:06.996 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-22T03:28:05.990615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":96371251,"num_objects":1468,"num_object_clones":0,"num_object_copies":4404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11831,"num_write_kb":95432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11776,"ondisk_log_size":11776,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":584656,"kb_used_data":584656,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208606256,"statfs":{"total":214211493888,"available":213612806144,"internally_reserved":0,"allocated":598687744,"data_stored":598687744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12673024,"num_objects":195,"num_object_clones":0,"num_object_copies":585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1547,"num_write_kb":12376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002105"},"pg_stats":[{"pgid":"5.f","version":"55'727","reported_seq":752,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.897397+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:28:01.897397+0000","last_peered":"2024-09-22T03:28:01.897397+0000","last_clean":"2024-09-22T03:28:01.897397+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:28:01.897397+0000","last_undegraded":"2024-09-22T03:28:01.897397+0000","last_fullsized":"2024-09-22T03:28:01.897397+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"55'847","reported_seq":872,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.798686+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:28:01.798686+0000","last_peered":"2024-09-22T03:28:01.798686+0000","last_clean":"2024-09-22T03:28:01.798686+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:28:01.798686+0000","last_undegraded":"2024-09-22T03:28:01.798686+0000","last_fullsized":"2024-09-22T03:28:01.798686+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"55'751","reported_seq":776,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.250672+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:28:01.250672+0000","last_peered":"2024-09-22T03:28:01.250672+0000","last_clean":"2024-09-22T03:28:01.250672+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:28:01.250672+0000","last_undegraded":"2024-09-22T03:28:01.250672+0000","last_fullsized":"2024-09-22T03:28:01.250672+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"55'735","reported_seq":760,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.414664+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:28:01.414664+0000","last_peered":"2024-09-22T03:28:01.414664+0000","last_clean":"2024-09-22T03:28:01.414664+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:28:01.414664+0000","last_undegraded":"2024-09-22T03:28:01.414664+0000","last_fullsized":"2024-09-22T03:28:01.414664+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"55'703","reported_seq":728,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.075643+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:28:00.075643+0000","last_peered":"2024-09-22T03:28:00.075643+0000","last_clean":"2024-09-22T03:28:00.075643+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:28:00.075643+0000","last_undegraded":"2024-09-22T03:28:00.075643+0000","last_fullsized":"2024-09-22T03:28:00.075643+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"55'720","reported_seq":745,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.513840+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:28:01.513840+0000","last_peered":"2024-09-22T03:28:01.513840+0000","last_clean":"2024-09-22T03:28:01.513840+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:28:01.513840+0000","last_undegraded":"2024-09-22T03:28:01.513840+0000","last_fullsized":"2024-09-22T03:28:01.513840+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"55'696","reported_seq":721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.596059+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:28:00.596059+0000","last_peered":"2024-09-22T03:28:00.596059+0000","last_clean":"2024-09-22T03:28:00.596059+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:28:00.596059+0000","last_undegraded":"2024-09-22T03:28:00.596059+0000","last_fullsized":"2024-09-22T03:28:00.596059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5709824,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":697,"num_write_kb":5576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"55'599","reported_seq":624,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.849826+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:28:01.849826+0000","last_peered":"2024-09-22T03:28:01.849826+0000","last_clean":"2024-09-22T03:28:01.849826+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:28:01.849826+0000","last_undegraded":"2024-09-22T03:28:01.849826+0000","last_fullsized":"2024-09-22T03:28:01.849826+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"55'791","reported_seq":816,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.690988+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:28:01.690988+0000","last_peered":"2024-09-22T03:28:01.690988+0000","last_clean":"2024-09-22T03:28:01.690988+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:28:01.690988+0000","last_undegraded":"2024-09-22T03:28:01.690988+0000","last_fullsized":"2024-09-22T03:28:01.690988+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.744310+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:14.744310+0000","last_peered":"2024-09-22T03:27:14.744310+0000","last_clean":"2024-09-22T03:27:14.744310+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:14.744310+0000","last_undegraded":"2024-09-22T03:27:14.744310+0000","last_fullsized":"2024-09-22T03:27:14.744310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":134,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.772027+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:14.772027+0000","last_peered":"2024-09-22T03:27:14.772027+0000","last_clean":"2024-09-22T03:27:14.772027+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:14.772027+0000","last_undegraded":"2024-09-22T03:27:14.772027+0000","last_fullsized":"2024-09-22T03:27:14.772027+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"55'695","reported_seq":720,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.571909+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:28:01.571909+0000","last_peered":"2024-09-22T03:28:01.571909+0000","last_clean":"2024-09-22T03:28:01.571909+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:28:01.571909+0000","last_undegraded":"2024-09-22T03:28:01.571909+0000","last_fullsized":"2024-09-22T03:28:01.571909+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.752756+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:14.752756+0000","last_peered":"2024-09-22T03:27:14.752756+0000","last_clean":"2024-09-22T03:27:14.752756+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:14.752756+0000","last_undegraded":"2024-09-22T03:27:14.752756+0000","last_fullsized":"2024-09-22T03:27:14.752756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"55'695","reported_seq":720,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.630464+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:28:00.630464+0000","last_peered":"2024-09-22T03:28:00.630464+0000","last_clean":"2024-09-22T03:28:00.630464+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:28:00.630464+0000","last_undegraded":"2024-09-22T03:28:00.630464+0000","last_fullsized":"2024-09-22T03:28:00.630464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.768492+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.768492+0000","last_peered":"2024-09-22T03:27:14.768492+0000","last_clean":"2024-09-22T03:27:14.768492+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:14.768492+0000","last_undegraded":"2024-09-22T03:27:14.768492+0000","last_fullsized":"2024-09-22T03:27:14.768492+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"55'863","reported_seq":888,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:00.420629+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:28:00.420629+0000","last_peered":"2024-09-22T03:28:00.420629+0000","last_clean":"2024-09-22T03:28:00.420629+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:28:00.420629+0000","last_undegraded":"2024-09-22T03:28:00.420629+0000","last_fullsized":"2024-09-22T03:28:00.420629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":135,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.750942+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.750942+0000","last_peered":"2024-09-22T03:27:14.750942+0000","last_clean":"2024-09-22T03:27:14.750942+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:14.750942+0000","last_undegraded":"2024-09-22T03:27:14.750942+0000","last_fullsized":"2024-09-22T03:27:14.750942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"55'665","reported_seq":690,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.912903+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:28:01.912903+0000","last_peered":"2024-09-22T03:28:01.912903+0000","last_clean":"2024-09-22T03:28:01.912903+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:28:01.912903+0000","last_undegraded":"2024-09-22T03:28:01.912903+0000","last_fullsized":"2024-09-22T03:28:01.912903+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5455872,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":666,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.756438+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.756438+0000","last_peered":"2024-09-22T03:27:14.756438+0000","last_clean":"2024-09-22T03:27:14.756438+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:14.756438+0000","last_undegraded":"2024-09-22T03:27:14.756438+0000","last_fullsized":"2024-09-22T03:27:14.756438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"55'695","reported_seq":720,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.636002+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:28:01.636002+0000","last_peered":"2024-09-22T03:28:01.636002+0000","last_clean":"2024-09-22T03:28:01.636002+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:28:01.636002+0000","last_undegraded":"2024-09-22T03:28:01.636002+0000","last_fullsized":"2024-09-22T03:28:01.636002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.748396+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.748396+0000","last_peered":"2024-09-22T03:27:14.748396+0000","last_clean":"2024-09-22T03:27:14.748396+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:14.748396+0000","last_undegraded":"2024-09-22T03:27:14.748396+0000","last_fullsized":"2024-09-22T03:27:14.748396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"55'719","reported_seq":744,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.020162+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:28:01.020162+0000","last_peered":"2024-09-22T03:28:01.020162+0000","last_clean":"2024-09-22T03:28:01.020162+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:28:01.020162+0000","last_undegraded":"2024-09-22T03:28:01.020162+0000","last_fullsized":"2024-09-22T03:28:01.020162+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.753346+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:14.753346+0000","last_peered":"2024-09-22T03:27:14.753346+0000","last_clean":"2024-09-22T03:27:14.753346+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:14.753346+0000","last_undegraded":"2024-09-22T03:27:14.753346+0000","last_fullsized":"2024-09-22T03:27:14.753346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"55'775","reported_seq":800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:01.741811+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:28:01.741811+0000","last_peered":"2024-09-22T03:28:01.741811+0000","last_clean":"2024-09-22T03:28:01.741811+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:28:01.741811+0000","last_undegraded":"2024-09-22T03:28:01.741811+0000","last_fullsized":"2024-09-22T03:28:01.741811+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.771801+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:14.771801+0000","last_peered":"2024-09-22T03:27:14.771801+0000","last_clean":"2024-09-22T03:27:14.771801+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:14.771801+0000","last_undegraded":"2024-09-22T03:27:14.771801+0000","last_fullsized":"2024-09-22T03:27:14.771801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":95780864,"num_objects":1464,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":93536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213605924864,"internally_reserved":0,"allocated":605569024,"data_stored":605569024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11676,"ondisk_log_size":11676,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213605924864,"internally_reserved":0,"allocated":605569024,"data_stored":605569024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53412106240,"internally_reserved":0,"allocated":140767232,"data_stored":140767232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":137388,"kb_used_data":137388,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52160340,"statfs":{"total":53552873472,"available":53412188160,"internally_reserved":0,"allocated":140685312,"data_stored":140685312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":170848,"kb_used_data":170848,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52126880,"statfs":{"total":53552873472,"available":53377925120,"internally_reserved":0,"allocated":174948352,"data_stored":174948352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":119876,"kb_used_data":119876,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52177852,"statfs":{"total":53552873472,"available":53430120448,"internally_reserved":0,"allocated":122753024,"data_stored":122753024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":156544,"kb_used_data":156544,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52141184,"statfs":{"total":53552873472,"available":53392572416,"internally_reserved":0,"allocated":160301056,"data_stored":160301056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53412106240,"internally_reserved":0,"allocated":140767232,"data_stored":140767232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53390053376,"internally_reserved":0,"allocated":162820096,"data_stored":162820096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53428510720,"internally_reserved":0,"allocated":124362752,"data_stored":124362752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53375254528,"internally_reserved":0,"allocated":177618944,"data_stored":177618944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53412106240,"internally_reserved":0,"allocated":140767232,"data_stored":140767232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53390053376,"internally_reserved":0,"allocated":162820096,"data_stored":162820096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53428510720,"internally_reserved":0,"allocated":124362752,"data_stored":124362752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53375254528,"internally_reserved":0,"allocated":177618944,"data_stored":177618944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53412106240,"internally_reserved":0,"allocated":140767232,"data_stored":140767232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:07.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 12652 12636 1.29882 1.32812 0.0938606 0.096171 2024-09-22T03:28:08.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:08.306+0000 7fc86126c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:08.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:08.313+0000 7fc86126c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:08.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 12814 12798 1.29838 1.26562 0.0539698 0.0962006 2024-09-22T03:28:09.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 15 12973 12958 1.29776 1.25 0.0883163 0.0962536 2024-09-22T03:28:10.428 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:10.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:10.566+0000 7fcb2c10e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:10.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:10.572+0000 7fcb2c10e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:10.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 13131 13115 1.29686 1.22656 0.185738 0.0963037 2024-09-22T03:28:11.848 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:28:11.848351+0000 min lat: 0.0117735 max lat: 0.474035 avg lat: 0.0963097 2024-09-22T03:28:11.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:28:11.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 13297 13281 1.29686 1.29688 0.121398 0.0963097 2024-09-22T03:28:11.998 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:11.998 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:11.998 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:28:11.998 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:28:12.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.141+0000 7f1d0926c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.150+0000 7f1d0926c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:12.165 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:12.166 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:12.166 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:12.168 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:12.168 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:12.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:28:12.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.287+0000 7fc3ae397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.296+0000 7fc3ae397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.342 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:12.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:12.346 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:12.346 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:28:12.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.495+0000 7fe340e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.503+0000 7fe340e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:12.527 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:12.527 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:12.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:28:12.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.645+0000 7f8379ec3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.658+0000 7f8379ec3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.679 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:12.679 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:12.679 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:12.679 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:12.679 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:12.680 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:12.680 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:12.680 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:12.680 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:12.680 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:12.680 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:12.680 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:12.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:12.683 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:12.781 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:12.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 13429 13413 1.29358 1.03125 0.0734699 0.0965473 2024-09-22T03:28:12.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.906+0000 7fdbdaad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:12.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:12.912+0000 7fdbdaad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:13.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 13590 13574 1.29314 1.25781 0.0532425 0.0965731 2024-09-22T03:28:14.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 13734 13718 1.29111 1.125 0.0549468 0.0966881 2024-09-22T03:28:15.059 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:15.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:15.186+0000 7f1cb1397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:15.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:15.191+0000 7f1cb1397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:15.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 13890 13874 1.29025 1.21875 0.0831579 0.0968252 2024-09-22T03:28:16.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 14022 14006 1.2872 1.03125 0.107508 0.0970056 2024-09-22T03:28:17.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:17.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:17.471+0000 7fc08d4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:17.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:17.477+0000 7fc08d4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:17.683 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:17.683 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:17.684 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:17.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:17.818+0000 7f61ae8fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:17.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:17.826+0000 7f61ae8fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:17.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 14180 14164 1.28659 1.23438 0.225002 0.0970598 2024-09-22T03:28:18.090 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:18.111 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-22T03:28:17.992737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":115581491,"num_objects":1760,"num_object_clones":0,"num_object_copies":5280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":14176,"num_write_kb":114192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14121,"ondisk_log_size":14121,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":703028,"kb_used_data":703028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208487884,"statfs":{"total":214211493888,"available":213491593216,"internally_reserved":0,"allocated":719900672,"data_stored":719900672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19210240,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2345,"num_write_kb":18760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002122"},"pg_stats":[{"pgid":"5.f","version":"55'911","reported_seq":936,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.903113+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:28:16.903113+0000","last_peered":"2024-09-22T03:28:16.903113+0000","last_clean":"2024-09-22T03:28:16.903113+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:28:16.903113+0000","last_undegraded":"2024-09-22T03:28:16.903113+0000","last_fullsized":"2024-09-22T03:28:16.903113+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"55'975","reported_seq":1000,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:15.088334+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:28:15.088334+0000","last_peered":"2024-09-22T03:28:15.088334+0000","last_clean":"2024-09-22T03:28:15.088334+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:28:15.088334+0000","last_undegraded":"2024-09-22T03:28:15.088334+0000","last_fullsized":"2024-09-22T03:28:15.088334+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7995392,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"55'975","reported_seq":1000,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:15.981576+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:28:15.981576+0000","last_peered":"2024-09-22T03:28:15.981576+0000","last_clean":"2024-09-22T03:28:15.981576+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:28:15.981576+0000","last_undegraded":"2024-09-22T03:28:15.981576+0000","last_fullsized":"2024-09-22T03:28:15.981576+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7995392,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"55'839","reported_seq":864,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.618275+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:28:16.618275+0000","last_peered":"2024-09-22T03:28:16.618275+0000","last_clean":"2024-09-22T03:28:16.618275+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:28:16.618275+0000","last_undegraded":"2024-09-22T03:28:16.618275+0000","last_fullsized":"2024-09-22T03:28:16.618275+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":839,"log_dups_size":0,"ondisk_log_size":839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881280,"num_objects":105,"num_object_clones":0,"num_object_copies":315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"55'883","reported_seq":908,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.942916+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:28:16.942916+0000","last_peered":"2024-09-22T03:28:16.942916+0000","last_clean":"2024-09-22T03:28:16.942916+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:28:16.942916+0000","last_undegraded":"2024-09-22T03:28:16.942916+0000","last_fullsized":"2024-09-22T03:28:16.942916+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7241728,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":884,"num_write_kb":7072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"55'863","reported_seq":888,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.464331+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:28:16.464331+0000","last_peered":"2024-09-22T03:28:16.464331+0000","last_clean":"2024-09-22T03:28:16.464331+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:28:16.464331+0000","last_undegraded":"2024-09-22T03:28:16.464331+0000","last_fullsized":"2024-09-22T03:28:16.464331+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"55'815","reported_seq":840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:14.407064+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:28:14.407064+0000","last_peered":"2024-09-22T03:28:14.407064+0000","last_clean":"2024-09-22T03:28:14.407064+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:28:14.407064+0000","last_undegraded":"2024-09-22T03:28:14.407064+0000","last_fullsized":"2024-09-22T03:28:14.407064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"55'759","reported_seq":784,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.855309+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:28:16.855309+0000","last_peered":"2024-09-22T03:28:16.855309+0000","last_clean":"2024-09-22T03:28:16.855309+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:28:16.855309+0000","last_undegraded":"2024-09-22T03:28:16.855309+0000","last_fullsized":"2024-09-22T03:28:16.855309+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"55'927","reported_seq":952,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.505484+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:28:16.505484+0000","last_peered":"2024-09-22T03:28:16.505484+0000","last_clean":"2024-09-22T03:28:16.505484+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:28:16.505484+0000","last_undegraded":"2024-09-22T03:28:16.505484+0000","last_fullsized":"2024-09-22T03:28:16.505484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.744310+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:14.744310+0000","last_peered":"2024-09-22T03:27:14.744310+0000","last_clean":"2024-09-22T03:27:14.744310+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:14.744310+0000","last_undegraded":"2024-09-22T03:27:14.744310+0000","last_fullsized":"2024-09-22T03:27:14.744310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":134,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.772027+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:14.772027+0000","last_peered":"2024-09-22T03:27:14.772027+0000","last_clean":"2024-09-22T03:27:14.772027+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:14.772027+0000","last_undegraded":"2024-09-22T03:27:14.772027+0000","last_fullsized":"2024-09-22T03:27:14.772027+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"55'863","reported_seq":888,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:15.996413+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:28:15.996413+0000","last_peered":"2024-09-22T03:28:15.996413+0000","last_clean":"2024-09-22T03:28:15.996413+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:28:15.996413+0000","last_undegraded":"2024-09-22T03:28:15.996413+0000","last_fullsized":"2024-09-22T03:28:15.996413+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.752756+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:14.752756+0000","last_peered":"2024-09-22T03:27:14.752756+0000","last_clean":"2024-09-22T03:27:14.752756+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:14.752756+0000","last_undegraded":"2024-09-22T03:27:14.752756+0000","last_fullsized":"2024-09-22T03:27:14.752756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"55'847","reported_seq":872,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.362522+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:28:16.362522+0000","last_peered":"2024-09-22T03:28:16.362522+0000","last_clean":"2024-09-22T03:28:16.362522+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:28:16.362522+0000","last_undegraded":"2024-09-22T03:28:16.362522+0000","last_fullsized":"2024-09-22T03:28:16.362522+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.768492+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.768492+0000","last_peered":"2024-09-22T03:27:14.768492+0000","last_clean":"2024-09-22T03:27:14.768492+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:14.768492+0000","last_undegraded":"2024-09-22T03:27:14.768492+0000","last_fullsized":"2024-09-22T03:27:14.768492+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"55'1000","reported_seq":1025,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:15.551286+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:28:15.551286+0000","last_peered":"2024-09-22T03:28:15.551286+0000","last_clean":"2024-09-22T03:28:15.551286+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:28:15.551286+0000","last_undegraded":"2024-09-22T03:28:15.551286+0000","last_fullsized":"2024-09-22T03:28:15.551286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1000,"log_dups_size":0,"ondisk_log_size":1000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8200192,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1001,"num_write_kb":8008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":135,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.750942+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.750942+0000","last_peered":"2024-09-22T03:27:14.750942+0000","last_clean":"2024-09-22T03:27:14.750942+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:14.750942+0000","last_undegraded":"2024-09-22T03:27:14.750942+0000","last_fullsized":"2024-09-22T03:27:14.750942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"55'751","reported_seq":776,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:15.401021+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:28:15.401021+0000","last_peered":"2024-09-22T03:28:15.401021+0000","last_clean":"2024-09-22T03:28:15.401021+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:28:15.401021+0000","last_undegraded":"2024-09-22T03:28:15.401021+0000","last_fullsized":"2024-09-22T03:28:15.401021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.756438+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.756438+0000","last_peered":"2024-09-22T03:27:14.756438+0000","last_clean":"2024-09-22T03:27:14.756438+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:14.756438+0000","last_undegraded":"2024-09-22T03:27:14.756438+0000","last_fullsized":"2024-09-22T03:27:14.756438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"55'799","reported_seq":824,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:14.475698+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:28:14.475698+0000","last_peered":"2024-09-22T03:28:14.475698+0000","last_clean":"2024-09-22T03:28:14.475698+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:28:14.475698+0000","last_undegraded":"2024-09-22T03:28:14.475698+0000","last_fullsized":"2024-09-22T03:28:14.475698+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.748396+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.748396+0000","last_peered":"2024-09-22T03:27:14.748396+0000","last_clean":"2024-09-22T03:27:14.748396+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:14.748396+0000","last_undegraded":"2024-09-22T03:27:14.748396+0000","last_fullsized":"2024-09-22T03:27:14.748396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"55'879","reported_seq":904,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.554834+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:28:16.554834+0000","last_peered":"2024-09-22T03:28:16.554834+0000","last_clean":"2024-09-22T03:28:16.554834+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:28:16.554834+0000","last_undegraded":"2024-09-22T03:28:16.554834+0000","last_fullsized":"2024-09-22T03:28:16.554834+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.753346+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:14.753346+0000","last_peered":"2024-09-22T03:27:14.753346+0000","last_clean":"2024-09-22T03:27:14.753346+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:14.753346+0000","last_undegraded":"2024-09-22T03:27:14.753346+0000","last_fullsized":"2024-09-22T03:27:14.753346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"55'935","reported_seq":960,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:16.704108+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:28:16.704108+0000","last_peered":"2024-09-22T03:28:16.704108+0000","last_clean":"2024-09-22T03:28:16.704108+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:28:16.704108+0000","last_undegraded":"2024-09-22T03:28:16.704108+0000","last_fullsized":"2024-09-22T03:28:16.704108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.771801+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:14.771801+0000","last_peered":"2024-09-22T03:27:14.771801+0000","last_clean":"2024-09-22T03:27:14.771801+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:14.771801+0000","last_undegraded":"2024-09-22T03:27:14.771801+0000","last_fullsized":"2024-09-22T03:27:14.771801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":114991104,"num_objects":1756,"num_object_clones":0,"num_object_copies":5268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14037,"num_write_kb":112296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213485944832,"internally_reserved":0,"allocated":725549056,"data_stored":725549056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14021,"ondisk_log_size":14021,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213485944832,"internally_reserved":0,"allocated":725549056,"data_stored":725549056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53384912896,"internally_reserved":0,"allocated":167960576,"data_stored":167960576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":164016,"kb_used_data":164016,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52133712,"statfs":{"total":53552873472,"available":53384921088,"internally_reserved":0,"allocated":167952384,"data_stored":167952384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":205552,"kb_used_data":205552,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52092176,"statfs":{"total":53552873472,"available":53342388224,"internally_reserved":0,"allocated":210485248,"data_stored":210485248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":142944,"kb_used_data":142944,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52154784,"statfs":{"total":53552873472,"available":53406498816,"internally_reserved":0,"allocated":146374656,"data_stored":146374656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":190516,"kb_used_data":190516,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52107212,"statfs":{"total":53552873472,"available":53357785088,"internally_reserved":0,"allocated":195088384,"data_stored":195088384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53384912896,"internally_reserved":0,"allocated":167960576,"data_stored":167960576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53355663360,"internally_reserved":0,"allocated":197210112,"data_stored":197210112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53405364224,"internally_reserved":0,"allocated":147509248,"data_stored":147509248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53340004352,"internally_reserved":0,"allocated":212869120,"data_stored":212869120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53384912896,"internally_reserved":0,"allocated":167960576,"data_stored":167960576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53355663360,"internally_reserved":0,"allocated":197210112,"data_stored":197210112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53405364224,"internally_reserved":0,"allocated":147509248,"data_stored":147509248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53340004352,"internally_reserved":0,"allocated":212869120,"data_stored":212869120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53384912896,"internally_reserved":0,"allocated":167960576,"data_stored":167960576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:18.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 14342 14326 1.28634 1.26562 0.0814899 0.0971187 2024-09-22T03:28:19.706 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:19.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:19.830+0000 7f2730797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:19.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:19.835+0000 7f2730797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:19.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 14492 14476 1.28504 1.17188 0.0696926 0.0971935 2024-09-22T03:28:20.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 14645 14629 1.28403 1.19531 0.0933896 0.0972797 2024-09-22T03:28:21.849 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 15 14806 14791 1.28383 1.26562 0.146522 0.0972791 2024-09-22T03:28:22.013 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:22.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:22.112+0000 7fcf3c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:22.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:22.117+0000 7fcf3c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.0972 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 14806 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.28386 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.125045 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.60156 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 1.03125 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 164 2024-09-22T03:28:22.850 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 16.0057 2024-09-22T03:28:22.851 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 205 2024-09-22T03:28:22.851 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 132 2024-09-22T03:28:22.851 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.0973098 2024-09-22T03:28:22.851 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.052285 2024-09-22T03:28:22.851 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.474035 2024-09-22T03:28:22.851 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0117735 2024-09-22T03:28:22.872 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:28:23.113 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:23.113 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:23.113 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:23.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:23.254+0000 7f8cea917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:23.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:23.263+0000 7f8cea917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:23.536 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:23.536 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:23.559 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-22T03:28:21.993516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121881139,"num_objects":1855,"num_object_clones":0,"num_object_copies":5565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":14945,"num_write_kb":120344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14890,"ondisk_log_size":14890,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":740976,"kb_used_data":740976,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208449936,"statfs":{"total":214211493888,"available":213452734464,"internally_reserved":0,"allocated":758759424,"data_stored":758759424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18907136,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2308,"num_write_kb":18464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002247"},"pg_stats":[{"pgid":"5.f","version":"55'967","reported_seq":992,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.792121+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:28:21.792121+0000","last_peered":"2024-09-22T03:28:21.792121+0000","last_clean":"2024-09-22T03:28:21.792121+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:28:21.792121+0000","last_undegraded":"2024-09-22T03:28:21.792121+0000","last_fullsized":"2024-09-22T03:28:21.792121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"55'1015","reported_seq":1040,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.124839+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:28:21.124839+0000","last_peered":"2024-09-22T03:28:21.124839+0000","last_clean":"2024-09-22T03:28:21.124839+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:28:21.124839+0000","last_undegraded":"2024-09-22T03:28:21.124839+0000","last_fullsized":"2024-09-22T03:28:21.124839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"55'1023","reported_seq":1048,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.606066+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:28:21.606066+0000","last_peered":"2024-09-22T03:28:21.606066+0000","last_clean":"2024-09-22T03:28:21.606066+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:28:21.606066+0000","last_undegraded":"2024-09-22T03:28:21.606066+0000","last_fullsized":"2024-09-22T03:28:21.606066+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1023,"log_dups_size":0,"ondisk_log_size":1023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":128,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1024,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"55'901","reported_seq":926,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.838136+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:28:21.838136+0000","last_peered":"2024-09-22T03:28:21.838136+0000","last_clean":"2024-09-22T03:28:21.838136+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:28:21.838136+0000","last_undegraded":"2024-09-22T03:28:21.838136+0000","last_fullsized":"2024-09-22T03:28:21.838136+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7389184,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":902,"num_write_kb":7216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"55'943","reported_seq":968,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.375179+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:28:21.375179+0000","last_peered":"2024-09-22T03:28:21.375179+0000","last_clean":"2024-09-22T03:28:21.375179+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:28:21.375179+0000","last_undegraded":"2024-09-22T03:28:21.375179+0000","last_fullsized":"2024-09-22T03:28:21.375179+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7733248,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":944,"num_write_kb":7552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"55'895","reported_seq":920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.335448+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:28:21.335448+0000","last_peered":"2024-09-22T03:28:21.335448+0000","last_clean":"2024-09-22T03:28:21.335448+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:28:21.335448+0000","last_undegraded":"2024-09-22T03:28:21.335448+0000","last_fullsized":"2024-09-22T03:28:21.335448+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"55'887","reported_seq":912,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.436158+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:28:21.436158+0000","last_peered":"2024-09-22T03:28:21.436158+0000","last_clean":"2024-09-22T03:28:21.436158+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:28:21.436158+0000","last_undegraded":"2024-09-22T03:28:21.436158+0000","last_fullsized":"2024-09-22T03:28:21.436158+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7274496,"num_objects":111,"num_object_clones":0,"num_object_copies":333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"55'815","reported_seq":840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.027237+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:28:21.027237+0000","last_peered":"2024-09-22T03:28:21.027237+0000","last_clean":"2024-09-22T03:28:21.027237+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:28:21.027237+0000","last_undegraded":"2024-09-22T03:28:21.027237+0000","last_fullsized":"2024-09-22T03:28:21.027237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"55'999","reported_seq":1024,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.461922+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:28:21.461922+0000","last_peered":"2024-09-22T03:28:21.461922+0000","last_clean":"2024-09-22T03:28:21.461922+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:28:21.461922+0000","last_undegraded":"2024-09-22T03:28:21.461922+0000","last_fullsized":"2024-09-22T03:28:21.461922+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":125,"num_object_clones":0,"num_object_copies":375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1000,"num_write_kb":8000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.744310+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:14.744310+0000","last_peered":"2024-09-22T03:27:14.744310+0000","last_clean":"2024-09-22T03:27:14.744310+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:14.744310+0000","last_undegraded":"2024-09-22T03:27:14.744310+0000","last_fullsized":"2024-09-22T03:27:14.744310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":134,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.772027+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:14.772027+0000","last_peered":"2024-09-22T03:27:14.772027+0000","last_clean":"2024-09-22T03:27:14.772027+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:14.772027+0000","last_undegraded":"2024-09-22T03:27:14.772027+0000","last_fullsized":"2024-09-22T03:27:14.772027+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"55'911","reported_seq":936,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.728753+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:28:21.728753+0000","last_peered":"2024-09-22T03:28:21.728753+0000","last_clean":"2024-09-22T03:28:21.728753+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:28:21.728753+0000","last_undegraded":"2024-09-22T03:28:21.728753+0000","last_fullsized":"2024-09-22T03:28:21.728753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.752756+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:14.752756+0000","last_peered":"2024-09-22T03:27:14.752756+0000","last_clean":"2024-09-22T03:27:14.752756+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:14.752756+0000","last_undegraded":"2024-09-22T03:27:14.752756+0000","last_fullsized":"2024-09-22T03:27:14.752756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"55'879","reported_seq":904,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:20.619259+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:28:20.619259+0000","last_peered":"2024-09-22T03:28:20.619259+0000","last_clean":"2024-09-22T03:28:20.619259+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:28:20.619259+0000","last_undegraded":"2024-09-22T03:28:20.619259+0000","last_fullsized":"2024-09-22T03:28:20.619259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.768492+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.768492+0000","last_peered":"2024-09-22T03:27:14.768492+0000","last_clean":"2024-09-22T03:27:14.768492+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:14.768492+0000","last_undegraded":"2024-09-22T03:27:14.768492+0000","last_fullsized":"2024-09-22T03:27:14.768492+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"55'1063","reported_seq":1088,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.076734+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:28:21.076734+0000","last_peered":"2024-09-22T03:28:21.076734+0000","last_clean":"2024-09-22T03:28:21.076734+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:28:21.076734+0000","last_undegraded":"2024-09-22T03:28:21.076734+0000","last_fullsized":"2024-09-22T03:28:21.076734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":133,"num_object_clones":0,"num_object_copies":399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1064,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":135,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.750942+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.750942+0000","last_peered":"2024-09-22T03:27:14.750942+0000","last_clean":"2024-09-22T03:27:14.750942+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:14.750942+0000","last_undegraded":"2024-09-22T03:27:14.750942+0000","last_fullsized":"2024-09-22T03:27:14.750942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"55'783","reported_seq":808,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:20.043816+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:28:20.043816+0000","last_peered":"2024-09-22T03:28:20.043816+0000","last_clean":"2024-09-22T03:28:20.043816+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:28:20.043816+0000","last_undegraded":"2024-09-22T03:28:20.043816+0000","last_fullsized":"2024-09-22T03:28:20.043816+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.756438+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.756438+0000","last_peered":"2024-09-22T03:27:14.756438+0000","last_clean":"2024-09-22T03:27:14.756438+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:14.756438+0000","last_undegraded":"2024-09-22T03:27:14.756438+0000","last_fullsized":"2024-09-22T03:27:14.756438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"55'807","reported_seq":832,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:19.354352+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:28:19.354352+0000","last_peered":"2024-09-22T03:28:19.354352+0000","last_clean":"2024-09-22T03:28:19.354352+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:28:19.354352+0000","last_undegraded":"2024-09-22T03:28:19.354352+0000","last_fullsized":"2024-09-22T03:28:19.354352+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.748396+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.748396+0000","last_peered":"2024-09-22T03:27:14.748396+0000","last_clean":"2024-09-22T03:27:14.748396+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:14.748396+0000","last_undegraded":"2024-09-22T03:27:14.748396+0000","last_fullsized":"2024-09-22T03:27:14.748396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"55'927","reported_seq":952,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:21.532989+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:28:21.532989+0000","last_peered":"2024-09-22T03:28:21.532989+0000","last_clean":"2024-09-22T03:28:21.532989+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:28:21.532989+0000","last_undegraded":"2024-09-22T03:28:21.532989+0000","last_fullsized":"2024-09-22T03:28:21.532989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.753346+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:14.753346+0000","last_peered":"2024-09-22T03:27:14.753346+0000","last_clean":"2024-09-22T03:27:14.753346+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:14.753346+0000","last_undegraded":"2024-09-22T03:27:14.753346+0000","last_fullsized":"2024-09-22T03:27:14.753346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"55'975","reported_seq":1000,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:20.999782+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:28:20.999782+0000","last_peered":"2024-09-22T03:28:20.999782+0000","last_clean":"2024-09-22T03:28:20.999782+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:28:20.999782+0000","last_undegraded":"2024-09-22T03:28:20.999782+0000","last_fullsized":"2024-09-22T03:28:20.999782+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7995392,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.771801+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:14.771801+0000","last_peered":"2024-09-22T03:27:14.771801+0000","last_clean":"2024-09-22T03:27:14.771801+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:14.771801+0000","last_undegraded":"2024-09-22T03:27:14.771801+0000","last_fullsized":"2024-09-22T03:27:14.771801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":121290752,"num_objects":1851,"num_object_clones":0,"num_object_copies":5553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14806,"num_write_kb":118448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213447221248,"internally_reserved":0,"allocated":764272640,"data_stored":764272640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14790,"ondisk_log_size":14790,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213447221248,"internally_reserved":0,"allocated":764272640,"data_stored":764272640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53377011712,"internally_reserved":0,"allocated":175861760,"data_stored":175861760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":171740,"kb_used_data":171740,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125988,"statfs":{"total":53552873472,"available":53377011712,"internally_reserved":0,"allocated":175861760,"data_stored":175861760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":217380,"kb_used_data":217380,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52080348,"statfs":{"total":53552873472,"available":53330276352,"internally_reserved":0,"allocated":222597120,"data_stored":222597120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":151080,"kb_used_data":151080,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52146648,"statfs":{"total":53552873472,"available":53398167552,"internally_reserved":0,"allocated":154705920,"data_stored":154705920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":200776,"kb_used_data":200776,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52096952,"statfs":{"total":53552873472,"available":53347278848,"internally_reserved":0,"allocated":205594624,"data_stored":205594624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53377011712,"internally_reserved":0,"allocated":175861760,"data_stored":175861760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53345357824,"internally_reserved":0,"allocated":207515648,"data_stored":207515648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53396705280,"internally_reserved":0,"allocated":156168192,"data_stored":156168192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53328146432,"internally_reserved":0,"allocated":224727040,"data_stored":224727040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53377011712,"internally_reserved":0,"allocated":175861760,"data_stored":175861760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53345357824,"internally_reserved":0,"allocated":207515648,"data_stored":207515648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53396705280,"internally_reserved":0,"allocated":156168192,"data_stored":156168192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53328146432,"internally_reserved":0,"allocated":224727040,"data_stored":224727040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53377011712,"internally_reserved":0,"allocated":175861760,"data_stored":175861760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:24.187 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:24.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:24.324+0000 7fb024917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:24.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:24.332+0000 7fb024917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:26.618 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:26.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:26.769+0000 7f4b19cce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:26.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:26.776+0000 7f4b19cce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:28.562 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:28.562 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:28.562 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:28.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:28.687+0000 7fdba6117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:28.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:28.695+0000 7fdba6117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:28.954 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:28.975 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-22T03:28:27.994782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75874891,"num_objects":1154,"num_object_clones":0,"num_object_copies":3462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":15648,"num_write_kb":120345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15593,"ondisk_log_size":15593,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":536004,"kb_used_data":536004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208654908,"statfs":{"total":214211493888,"available":213662625792,"internally_reserved":0,"allocated":548868096,"data_stored":548868096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33718248,"num_objects":-514,"num_object_clones":0,"num_object_copies":-1542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2203,"num_write_kb":12001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002419"},"pg_stats":[{"pgid":"5.f","version":"55'1001","reported_seq":1026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.588646+0000","last_change":"2024-09-22T03:26:50.856500+0000","last_active":"2024-09-22T03:28:26.588646+0000","last_peered":"2024-09-22T03:28:26.588646+0000","last_clean":"2024-09-22T03:28:26.588646+0000","last_became_active":"2024-09-22T03:26:50.850418+0000","last_became_peered":"2024-09-22T03:26:50.850418+0000","last_unstale":"2024-09-22T03:28:26.588646+0000","last_undegraded":"2024-09-22T03:28:26.588646+0000","last_fullsized":"2024-09-22T03:28:26.588646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1002,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"55'1069","reported_seq":1094,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.847028+0000","last_change":"2024-09-22T03:26:50.899558+0000","last_active":"2024-09-22T03:28:26.847028+0000","last_peered":"2024-09-22T03:28:26.847028+0000","last_clean":"2024-09-22T03:28:26.847028+0000","last_became_active":"2024-09-22T03:26:50.894857+0000","last_became_peered":"2024-09-22T03:26:50.894857+0000","last_unstale":"2024-09-22T03:28:26.847028+0000","last_undegraded":"2024-09-22T03:28:26.847028+0000","last_fullsized":"2024-09-22T03:28:26.847028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1069,"log_dups_size":0,"ondisk_log_size":1069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1070,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"55'1074","reported_seq":1100,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.810674+0000","last_change":"2024-09-22T03:26:50.923248+0000","last_active":"2024-09-22T03:28:26.810674+0000","last_peered":"2024-09-22T03:28:26.810674+0000","last_clean":"2024-09-22T03:28:26.810674+0000","last_became_active":"2024-09-22T03:26:50.901817+0000","last_became_peered":"2024-09-22T03:26:50.901817+0000","last_unstale":"2024-09-22T03:28:26.810674+0000","last_undegraded":"2024-09-22T03:28:26.810674+0000","last_fullsized":"2024-09-22T03:28:26.810674+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111832,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1075,"num_write_kb":8193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"55'941","reported_seq":966,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.921230+0000","last_change":"2024-09-22T03:26:50.951494+0000","last_active":"2024-09-22T03:28:26.921230+0000","last_peered":"2024-09-22T03:28:26.921230+0000","last_clean":"2024-09-22T03:28:26.921230+0000","last_became_active":"2024-09-22T03:26:50.942018+0000","last_became_peered":"2024-09-22T03:26:50.942018+0000","last_unstale":"2024-09-22T03:28:26.921230+0000","last_undegraded":"2024-09-22T03:28:26.921230+0000","last_fullsized":"2024-09-22T03:28:26.921230+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":941,"log_dups_size":0,"ondisk_log_size":941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4767744,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":942,"num_write_kb":7216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"55'979","reported_seq":1004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.690219+0000","last_change":"2024-09-22T03:26:50.924279+0000","last_active":"2024-09-22T03:28:26.690219+0000","last_peered":"2024-09-22T03:28:26.690219+0000","last_clean":"2024-09-22T03:28:26.690219+0000","last_became_active":"2024-09-22T03:26:50.914888+0000","last_became_peered":"2024-09-22T03:26:50.914888+0000","last_unstale":"2024-09-22T03:28:26.690219+0000","last_undegraded":"2024-09-22T03:28:26.690219+0000","last_fullsized":"2024-09-22T03:28:26.690219+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":979,"log_dups_size":0,"ondisk_log_size":979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":980,"num_write_kb":7552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"55'939","reported_seq":964,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.653868+0000","last_change":"2024-09-22T03:26:50.886917+0000","last_active":"2024-09-22T03:28:26.653868+0000","last_peered":"2024-09-22T03:28:26.653868+0000","last_clean":"2024-09-22T03:28:26.653868+0000","last_became_active":"2024-09-22T03:26:50.856669+0000","last_became_peered":"2024-09-22T03:26:50.856669+0000","last_unstale":"2024-09-22T03:28:26.653868+0000","last_undegraded":"2024-09-22T03:28:26.653868+0000","last_fullsized":"2024-09-22T03:28:26.653868+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":940,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"55'929","reported_seq":954,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.776822+0000","last_change":"2024-09-22T03:26:50.888020+0000","last_active":"2024-09-22T03:28:26.776822+0000","last_peered":"2024-09-22T03:28:26.776822+0000","last_clean":"2024-09-22T03:28:26.776822+0000","last_became_active":"2024-09-22T03:26:50.883327+0000","last_became_peered":"2024-09-22T03:26:50.883327+0000","last_unstale":"2024-09-22T03:28:26.776822+0000","last_undegraded":"2024-09-22T03:28:26.776822+0000","last_fullsized":"2024-09-22T03:28:26.776822+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":929,"log_dups_size":0,"ondisk_log_size":929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":930,"num_write_kb":7104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"55'851","reported_seq":876,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.851991+0000","last_change":"2024-09-22T03:26:50.913102+0000","last_active":"2024-09-22T03:28:26.851991+0000","last_peered":"2024-09-22T03:28:26.851991+0000","last_clean":"2024-09-22T03:28:26.851991+0000","last_became_active":"2024-09-22T03:26:50.909016+0000","last_became_peered":"2024-09-22T03:26:50.909016+0000","last_unstale":"2024-09-22T03:28:26.851991+0000","last_undegraded":"2024-09-22T03:28:26.851991+0000","last_fullsized":"2024-09-22T03:28:26.851991+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":852,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"55'1046","reported_seq":1071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.842924+0000","last_change":"2024-09-22T03:26:50.911564+0000","last_active":"2024-09-22T03:28:26.842924+0000","last_peered":"2024-09-22T03:28:26.842924+0000","last_clean":"2024-09-22T03:28:26.842924+0000","last_became_active":"2024-09-22T03:26:50.907375+0000","last_became_peered":"2024-09-22T03:26:50.907375+0000","last_unstale":"2024-09-22T03:28:26.842924+0000","last_undegraded":"2024-09-22T03:28:26.842924+0000","last_fullsized":"2024-09-22T03:28:26.842924+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1046,"log_dups_size":0,"ondisk_log_size":1046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1047,"num_write_kb":8000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":268,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.744310+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:27:14.744310+0000","last_peered":"2024-09-22T03:27:14.744310+0000","last_clean":"2024-09-22T03:27:14.744310+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:27:14.744310+0000","last_undegraded":"2024-09-22T03:27:14.744310+0000","last_fullsized":"2024-09-22T03:27:14.744310+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":134,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.772027+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:27:14.772027+0000","last_peered":"2024-09-22T03:27:14.772027+0000","last_clean":"2024-09-22T03:27:14.772027+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:27:14.772027+0000","last_undegraded":"2024-09-22T03:27:14.772027+0000","last_fullsized":"2024-09-22T03:27:14.772027+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"55'946","reported_seq":971,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.588771+0000","last_change":"2024-09-22T03:26:50.844426+0000","last_active":"2024-09-22T03:28:26.588771+0000","last_peered":"2024-09-22T03:28:26.588771+0000","last_clean":"2024-09-22T03:28:26.588771+0000","last_became_active":"2024-09-22T03:26:50.839350+0000","last_became_peered":"2024-09-22T03:26:50.839350+0000","last_unstale":"2024-09-22T03:28:26.588771+0000","last_undegraded":"2024-09-22T03:28:26.588771+0000","last_fullsized":"2024-09-22T03:28:26.588771+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":947,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.752756+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:27:14.752756+0000","last_peered":"2024-09-22T03:27:14.752756+0000","last_clean":"2024-09-22T03:27:14.752756+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:27:14.752756+0000","last_undegraded":"2024-09-22T03:27:14.752756+0000","last_fullsized":"2024-09-22T03:27:14.752756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"55'926","reported_seq":951,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.678229+0000","last_change":"2024-09-22T03:26:50.943462+0000","last_active":"2024-09-22T03:28:26.678229+0000","last_peered":"2024-09-22T03:28:26.678229+0000","last_clean":"2024-09-22T03:28:26.678229+0000","last_became_active":"2024-09-22T03:26:50.938713+0000","last_became_peered":"2024-09-22T03:26:50.938713+0000","last_unstale":"2024-09-22T03:28:26.678229+0000","last_undegraded":"2024-09-22T03:28:26.678229+0000","last_fullsized":"2024-09-22T03:28:26.678229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":927,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.768492+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.768492+0000","last_peered":"2024-09-22T03:27:14.768492+0000","last_clean":"2024-09-22T03:27:14.768492+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:27:14.768492+0000","last_undegraded":"2024-09-22T03:27:14.768492+0000","last_fullsized":"2024-09-22T03:27:14.768492+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"55'1117","reported_seq":1142,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.884024+0000","last_change":"2024-09-22T03:26:50.873334+0000","last_active":"2024-09-22T03:28:26.884024+0000","last_peered":"2024-09-22T03:28:26.884024+0000","last_clean":"2024-09-22T03:28:26.884024+0000","last_became_active":"2024-09-22T03:26:50.869868+0000","last_became_peered":"2024-09-22T03:26:50.869868+0000","last_unstale":"2024-09-22T03:28:26.884024+0000","last_undegraded":"2024-09-22T03:28:26.884024+0000","last_fullsized":"2024-09-22T03:28:26.884024+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1118,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":135,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.750942+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:27:14.750942+0000","last_peered":"2024-09-22T03:27:14.750942+0000","last_clean":"2024-09-22T03:27:14.750942+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:27:14.750942+0000","last_undegraded":"2024-09-22T03:27:14.750942+0000","last_fullsized":"2024-09-22T03:27:14.750942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"55'821","reported_seq":846,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.819695+0000","last_change":"2024-09-22T03:26:50.883033+0000","last_active":"2024-09-22T03:28:26.819695+0000","last_peered":"2024-09-22T03:28:26.819695+0000","last_clean":"2024-09-22T03:28:26.819695+0000","last_became_active":"2024-09-22T03:26:50.842357+0000","last_became_peered":"2024-09-22T03:26:50.842357+0000","last_unstale":"2024-09-22T03:28:26.819695+0000","last_undegraded":"2024-09-22T03:28:26.819695+0000","last_fullsized":"2024-09-22T03:28:26.819695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":822,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.756438+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.756438+0000","last_peered":"2024-09-22T03:27:14.756438+0000","last_clean":"2024-09-22T03:27:14.756438+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:27:14.756438+0000","last_undegraded":"2024-09-22T03:27:14.756438+0000","last_fullsized":"2024-09-22T03:27:14.756438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"55'857","reported_seq":882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.624848+0000","last_change":"2024-09-22T03:26:50.909116+0000","last_active":"2024-09-22T03:28:26.624848+0000","last_peered":"2024-09-22T03:28:26.624848+0000","last_clean":"2024-09-22T03:28:26.624848+0000","last_became_active":"2024-09-22T03:26:50.904368+0000","last_became_peered":"2024-09-22T03:26:50.904368+0000","last_unstale":"2024-09-22T03:28:26.624848+0000","last_undegraded":"2024-09-22T03:28:26.624848+0000","last_fullsized":"2024-09-22T03:28:26.624848+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":858,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.748396+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:27:14.748396+0000","last_peered":"2024-09-22T03:27:14.748396+0000","last_clean":"2024-09-22T03:27:14.748396+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:27:14.748396+0000","last_undegraded":"2024-09-22T03:27:14.748396+0000","last_fullsized":"2024-09-22T03:27:14.748396+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"55'973","reported_seq":998,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.877087+0000","last_change":"2024-09-22T03:26:50.891684+0000","last_active":"2024-09-22T03:28:26.877087+0000","last_peered":"2024-09-22T03:28:26.877087+0000","last_clean":"2024-09-22T03:28:26.877087+0000","last_became_active":"2024-09-22T03:26:50.878863+0000","last_became_peered":"2024-09-22T03:26:50.878863+0000","last_unstale":"2024-09-22T03:28:26.877087+0000","last_undegraded":"2024-09-22T03:28:26.877087+0000","last_fullsized":"2024-09-22T03:28:26.877087+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":974,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.753346+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:27:14.753346+0000","last_peered":"2024-09-22T03:27:14.753346+0000","last_clean":"2024-09-22T03:27:14.753346+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:27:14.753346+0000","last_undegraded":"2024-09-22T03:27:14.753346+0000","last_fullsized":"2024-09-22T03:27:14.753346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"55'1024","reported_seq":1049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:28:26.833015+0000","last_change":"2024-09-22T03:26:50.927581+0000","last_active":"2024-09-22T03:28:26.833015+0000","last_peered":"2024-09-22T03:28:26.833015+0000","last_clean":"2024-09-22T03:28:26.833015+0000","last_became_active":"2024-09-22T03:26:50.924271+0000","last_became_peered":"2024-09-22T03:26:50.924271+0000","last_unstale":"2024-09-22T03:28:26.833015+0000","last_undegraded":"2024-09-22T03:28:26.833015+0000","last_fullsized":"2024-09-22T03:28:26.833015+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:26:49.758629+0000","last_clean_scrub_stamp":"2024-09-22T03:26:49.758629+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1025,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-22T03:27:14.771801+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:27:14.771801+0000","last_peered":"2024-09-22T03:27:14.771801+0000","last_clean":"2024-09-22T03:27:14.771801+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:27:14.771801+0000","last_undegraded":"2024-09-22T03:27:14.771801+0000","last_fullsized":"2024-09-22T03:27:14.771801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":75284504,"num_objects":1150,"num_object_clones":0,"num_object_copies":3450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15509,"num_write_kb":118449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213720207360,"internally_reserved":0,"allocated":491286528,"data_stored":491286528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15493,"ondisk_log_size":15493,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213720076288,"internally_reserved":0,"allocated":491417600,"data_stored":491417600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53441417216,"internally_reserved":0,"allocated":111456256,"data_stored":111456256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":123120,"kb_used_data":123120,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52174608,"statfs":{"total":53552873472,"available":53426798592,"internally_reserved":0,"allocated":126074880,"data_stored":126074880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":158604,"kb_used_data":158604,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52139124,"statfs":{"total":53552873472,"available":53390462976,"internally_reserved":0,"allocated":162410496,"data_stored":162410496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":108316,"kb_used_data":108316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52189412,"statfs":{"total":53552873472,"available":53441957888,"internally_reserved":0,"allocated":110915584,"data_stored":110915584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":145964,"kb_used_data":145964,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52151764,"statfs":{"total":53552873472,"available":53403406336,"internally_reserved":0,"allocated":149467136,"data_stored":149467136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53441417216,"internally_reserved":0,"allocated":111456256,"data_stored":111456256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53419343872,"internally_reserved":0,"allocated":133529600,"data_stored":133529600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53452509184,"internally_reserved":0,"allocated":100364288,"data_stored":100364288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53406806016,"internally_reserved":0,"allocated":146067456,"data_stored":146067456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53441417216,"internally_reserved":0,"allocated":111456256,"data_stored":111456256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":53552873472,"available":53419343872,"internally_reserved":0,"allocated":133529600,"data_stored":133529600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":53552873472,"available":53452509184,"internally_reserved":0,"allocated":100364288,"data_stored":100364288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":53552873472,"available":53406937088,"internally_reserved":0,"allocated":145936384,"data_stored":145936384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":53552873472,"available":53441417216,"internally_reserved":0,"allocated":111456256,"data_stored":111456256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:29.004 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:29.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:29.127+0000 7f63d362c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:29.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:29.133+0000 7f63d362c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:31.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:31.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:31.410+0000 7f6ae2f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:31.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:31.415+0000 7f6ae2f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:33.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:33.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:33.784+0000 7fd64202c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:33.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:33.789+0000 7fd64202c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:33.977 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:33.978 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:28:34.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:34.116+0000 7f378be4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:34.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:34.130+0000 7f378be4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:34.711 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 1851 objects 2024-09-22T03:28:34.711 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :11.8389 2024-09-22T03:28:34.847 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:28:34.865 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:28:34.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:34.972+0000 7f95e90b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:34.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:34.981+0000 7f95e90b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:35.512 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-22T03:28:35.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-22T03:28:35.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:35.613+0000 7fc191957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:35.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:35.624+0000 7fc191957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:35.851 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:28:35.851 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_2' removed 2024-09-22T03:28:35.874 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:28:35.875 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:28:35.875 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-22T03:28:35.875 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_3 16 2024-09-22T03:28:35.876 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:28:36.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:36.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:36.063+0000 7fb0d7e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:36.075+0000 7fb0d7e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:36.077+0000 7f52e3917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:36.090+0000 7f52e3917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:36.140+0000 7f6e70d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:36.145+0000 7f6e70d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:36.864 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:28:36.864 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_3' created 2024-09-22T03:28:36.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:28:36.883 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-22T03:28:37.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:37.032+0000 7fb146d03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:37.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:37.040+0000 7fb146d03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:37.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:37.032+0000 7f231d597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:37.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:37.045+0000 7f231d597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:37.870 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-22T03:28:37.870 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:28:37.889 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:28:37.890 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:38.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.062+0000 7f3b03517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.071+0000 7f3b03517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:38.256 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:38.256 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:38.268 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-22T03:28:37.996565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":28092,"kb_used_data":28092,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209162820,"statfs":{"total":214211493888,"available":214182727680,"internally_reserved":0,"allocated":28766208,"data_stored":28766208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26066968,"num_objects":-399,"num_object_clones":0,"num_object_copies":-1197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-16260,"num_write_kb":-118449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000647"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.938584+0000","last_change":"2024-09-22T03:28:36.938584+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.938584+0000","last_undegraded":"2024-09-22T03:28:36.938584+0000","last_fullsized":"2024-09-22T03:28:36.938584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.932214+0000","last_change":"2024-09-22T03:28:36.932214+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.932214+0000","last_undegraded":"2024-09-22T03:28:36.932214+0000","last_fullsized":"2024-09-22T03:28:36.932214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.917916+0000","last_change":"2024-09-22T03:28:36.917916+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.917916+0000","last_undegraded":"2024-09-22T03:28:36.917916+0000","last_fullsized":"2024-09-22T03:28:36.917916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927289+0000","last_change":"2024-09-22T03:28:36.927289+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927289+0000","last_undegraded":"2024-09-22T03:28:36.927289+0000","last_fullsized":"2024-09-22T03:28:36.927289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.904312+0000","last_change":"2024-09-22T03:28:36.904312+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.904312+0000","last_undegraded":"2024-09-22T03:28:36.904312+0000","last_fullsized":"2024-09-22T03:28:36.904312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927970+0000","last_change":"2024-09-22T03:28:36.927970+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927970+0000","last_undegraded":"2024-09-22T03:28:36.927970+0000","last_fullsized":"2024-09-22T03:28:36.927970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.912291+0000","last_change":"2024-09-22T03:28:36.912291+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.912291+0000","last_undegraded":"2024-09-22T03:28:36.912291+0000","last_fullsized":"2024-09-22T03:28:36.912291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927974+0000","last_change":"2024-09-22T03:28:36.927974+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927974+0000","last_undegraded":"2024-09-22T03:28:36.927974+0000","last_fullsized":"2024-09-22T03:28:36.927974+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.916162+0000","last_change":"2024-09-22T03:28:36.916162+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.916162+0000","last_undegraded":"2024-09-22T03:28:36.916162+0000","last_fullsized":"2024-09-22T03:28:36.916162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":277,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.879485+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:36.879485+0000","last_peered":"2024-09-22T03:28:36.879485+0000","last_clean":"2024-09-22T03:28:36.879485+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:36.879485+0000","last_undegraded":"2024-09-22T03:28:36.879485+0000","last_fullsized":"2024-09-22T03:28:36.879485+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.887714+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:36.887714+0000","last_peered":"2024-09-22T03:28:36.887714+0000","last_clean":"2024-09-22T03:28:36.887714+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:36.887714+0000","last_undegraded":"2024-09-22T03:28:36.887714+0000","last_fullsized":"2024-09-22T03:28:36.887714+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.886782+0000","last_change":"2024-09-22T03:28:36.886782+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.886782+0000","last_undegraded":"2024-09-22T03:28:36.886782+0000","last_fullsized":"2024-09-22T03:28:36.886782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.900203+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:36.900203+0000","last_peered":"2024-09-22T03:28:36.900203+0000","last_clean":"2024-09-22T03:28:36.900203+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:36.900203+0000","last_undegraded":"2024-09-22T03:28:36.900203+0000","last_fullsized":"2024-09-22T03:28:36.900203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.914604+0000","last_change":"2024-09-22T03:28:36.914604+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.914604+0000","last_undegraded":"2024-09-22T03:28:36.914604+0000","last_fullsized":"2024-09-22T03:28:36.914604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.886250+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:36.886250+0000","last_peered":"2024-09-22T03:28:36.886250+0000","last_clean":"2024-09-22T03:28:36.886250+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:36.886250+0000","last_undegraded":"2024-09-22T03:28:36.886250+0000","last_fullsized":"2024-09-22T03:28:36.886250+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.896833+0000","last_change":"2024-09-22T03:28:36.896833+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.896833+0000","last_undegraded":"2024-09-22T03:28:36.896833+0000","last_fullsized":"2024-09-22T03:28:36.896833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":144,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.898290+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:36.898290+0000","last_peered":"2024-09-22T03:28:36.898290+0000","last_clean":"2024-09-22T03:28:36.898290+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:36.898290+0000","last_undegraded":"2024-09-22T03:28:36.898290+0000","last_fullsized":"2024-09-22T03:28:36.898290+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.889697+0000","last_change":"2024-09-22T03:28:36.889697+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.889697+0000","last_undegraded":"2024-09-22T03:28:36.889697+0000","last_fullsized":"2024-09-22T03:28:36.889697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.885663+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:36.885663+0000","last_peered":"2024-09-22T03:28:36.885663+0000","last_clean":"2024-09-22T03:28:36.885663+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:36.885663+0000","last_undegraded":"2024-09-22T03:28:36.885663+0000","last_fullsized":"2024-09-22T03:28:36.885663+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.900131+0000","last_change":"2024-09-22T03:28:36.900131+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.900131+0000","last_undegraded":"2024-09-22T03:28:36.900131+0000","last_fullsized":"2024-09-22T03:28:36.900131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.881697+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:36.881697+0000","last_peered":"2024-09-22T03:28:36.881697+0000","last_clean":"2024-09-22T03:28:36.881697+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:36.881697+0000","last_undegraded":"2024-09-22T03:28:36.881697+0000","last_fullsized":"2024-09-22T03:28:36.881697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.891760+0000","last_change":"2024-09-22T03:28:36.891760+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.891760+0000","last_undegraded":"2024-09-22T03:28:36.891760+0000","last_fullsized":"2024-09-22T03:28:36.891760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.884986+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:36.884986+0000","last_peered":"2024-09-22T03:28:36.884986+0000","last_clean":"2024-09-22T03:28:36.884986+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:36.884986+0000","last_undegraded":"2024-09-22T03:28:36.884986+0000","last_fullsized":"2024-09-22T03:28:36.884986+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.905481+0000","last_change":"2024-09-22T03:28:36.905481+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.905481+0000","last_undegraded":"2024-09-22T03:28:36.905481+0000","last_fullsized":"2024-09-22T03:28:36.905481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.887122+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:36.887122+0000","last_peered":"2024-09-22T03:28:36.887122+0000","last_clean":"2024-09-22T03:28:36.887122+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:36.887122+0000","last_undegraded":"2024-09-22T03:28:36.887122+0000","last_fullsized":"2024-09-22T03:28:36.887122+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214202429440,"internally_reserved":0,"allocated":9064448,"data_stored":9064448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214202429440,"internally_reserved":0,"allocated":9064448,"data_stored":9064448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7272,"kb_used_data":7272,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290456,"statfs":{"total":53552873472,"available":53545426944,"internally_reserved":0,"allocated":7446528,"data_stored":7446528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":9828,"kb_used_data":9828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52287900,"statfs":{"total":53552873472,"available":53542809600,"internally_reserved":0,"allocated":10063872,"data_stored":10063872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2896,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294832,"statfs":{"total":53552873472,"available":53549907968,"internally_reserved":0,"allocated":2965504,"data_stored":2965504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8096,"kb_used_data":8096,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289632,"statfs":{"total":53552873472,"available":53544583168,"internally_reserved":0,"allocated":8290304,"data_stored":8290304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550366720,"internally_reserved":0,"allocated":2506752,"data_stored":2506752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551300608,"internally_reserved":0,"allocated":1572864,"data_stored":1572864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550211072,"internally_reserved":0,"allocated":2662400,"data_stored":2662400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53550366720,"internally_reserved":0,"allocated":2506752,"data_stored":2506752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53551300608,"internally_reserved":0,"allocated":1572864,"data_stored":1572864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53550211072,"internally_reserved":0,"allocated":2662400,"data_stored":2662400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:38.270 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:38.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.325+0000 7fc881397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.330+0000 7fc881397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.423+0000 7f92fc997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.430+0000 7f92fc997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.607 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:38.608 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:38.621 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-22T03:28:37.996565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":28092,"kb_used_data":28092,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209162820,"statfs":{"total":214211493888,"available":214182727680,"internally_reserved":0,"allocated":28766208,"data_stored":28766208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26066968,"num_objects":-399,"num_object_clones":0,"num_object_copies":-1197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-16260,"num_write_kb":-118449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000647"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.938584+0000","last_change":"2024-09-22T03:28:36.938584+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.938584+0000","last_undegraded":"2024-09-22T03:28:36.938584+0000","last_fullsized":"2024-09-22T03:28:36.938584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.932214+0000","last_change":"2024-09-22T03:28:36.932214+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.932214+0000","last_undegraded":"2024-09-22T03:28:36.932214+0000","last_fullsized":"2024-09-22T03:28:36.932214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.917916+0000","last_change":"2024-09-22T03:28:36.917916+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.917916+0000","last_undegraded":"2024-09-22T03:28:36.917916+0000","last_fullsized":"2024-09-22T03:28:36.917916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927289+0000","last_change":"2024-09-22T03:28:36.927289+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927289+0000","last_undegraded":"2024-09-22T03:28:36.927289+0000","last_fullsized":"2024-09-22T03:28:36.927289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.904312+0000","last_change":"2024-09-22T03:28:36.904312+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.904312+0000","last_undegraded":"2024-09-22T03:28:36.904312+0000","last_fullsized":"2024-09-22T03:28:36.904312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927970+0000","last_change":"2024-09-22T03:28:36.927970+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927970+0000","last_undegraded":"2024-09-22T03:28:36.927970+0000","last_fullsized":"2024-09-22T03:28:36.927970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.912291+0000","last_change":"2024-09-22T03:28:36.912291+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.912291+0000","last_undegraded":"2024-09-22T03:28:36.912291+0000","last_fullsized":"2024-09-22T03:28:36.912291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927974+0000","last_change":"2024-09-22T03:28:36.927974+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927974+0000","last_undegraded":"2024-09-22T03:28:36.927974+0000","last_fullsized":"2024-09-22T03:28:36.927974+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.916162+0000","last_change":"2024-09-22T03:28:36.916162+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.916162+0000","last_undegraded":"2024-09-22T03:28:36.916162+0000","last_fullsized":"2024-09-22T03:28:36.916162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":277,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.879485+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:36.879485+0000","last_peered":"2024-09-22T03:28:36.879485+0000","last_clean":"2024-09-22T03:28:36.879485+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:36.879485+0000","last_undegraded":"2024-09-22T03:28:36.879485+0000","last_fullsized":"2024-09-22T03:28:36.879485+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.887714+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:36.887714+0000","last_peered":"2024-09-22T03:28:36.887714+0000","last_clean":"2024-09-22T03:28:36.887714+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:36.887714+0000","last_undegraded":"2024-09-22T03:28:36.887714+0000","last_fullsized":"2024-09-22T03:28:36.887714+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.886782+0000","last_change":"2024-09-22T03:28:36.886782+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.886782+0000","last_undegraded":"2024-09-22T03:28:36.886782+0000","last_fullsized":"2024-09-22T03:28:36.886782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.900203+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:36.900203+0000","last_peered":"2024-09-22T03:28:36.900203+0000","last_clean":"2024-09-22T03:28:36.900203+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:36.900203+0000","last_undegraded":"2024-09-22T03:28:36.900203+0000","last_fullsized":"2024-09-22T03:28:36.900203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.914604+0000","last_change":"2024-09-22T03:28:36.914604+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.914604+0000","last_undegraded":"2024-09-22T03:28:36.914604+0000","last_fullsized":"2024-09-22T03:28:36.914604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.886250+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:36.886250+0000","last_peered":"2024-09-22T03:28:36.886250+0000","last_clean":"2024-09-22T03:28:36.886250+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:36.886250+0000","last_undegraded":"2024-09-22T03:28:36.886250+0000","last_fullsized":"2024-09-22T03:28:36.886250+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.896833+0000","last_change":"2024-09-22T03:28:36.896833+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.896833+0000","last_undegraded":"2024-09-22T03:28:36.896833+0000","last_fullsized":"2024-09-22T03:28:36.896833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":144,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.898290+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:36.898290+0000","last_peered":"2024-09-22T03:28:36.898290+0000","last_clean":"2024-09-22T03:28:36.898290+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:36.898290+0000","last_undegraded":"2024-09-22T03:28:36.898290+0000","last_fullsized":"2024-09-22T03:28:36.898290+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.889697+0000","last_change":"2024-09-22T03:28:36.889697+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.889697+0000","last_undegraded":"2024-09-22T03:28:36.889697+0000","last_fullsized":"2024-09-22T03:28:36.889697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.885663+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:36.885663+0000","last_peered":"2024-09-22T03:28:36.885663+0000","last_clean":"2024-09-22T03:28:36.885663+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:36.885663+0000","last_undegraded":"2024-09-22T03:28:36.885663+0000","last_fullsized":"2024-09-22T03:28:36.885663+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.900131+0000","last_change":"2024-09-22T03:28:36.900131+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.900131+0000","last_undegraded":"2024-09-22T03:28:36.900131+0000","last_fullsized":"2024-09-22T03:28:36.900131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.881697+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:36.881697+0000","last_peered":"2024-09-22T03:28:36.881697+0000","last_clean":"2024-09-22T03:28:36.881697+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:36.881697+0000","last_undegraded":"2024-09-22T03:28:36.881697+0000","last_fullsized":"2024-09-22T03:28:36.881697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.891760+0000","last_change":"2024-09-22T03:28:36.891760+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.891760+0000","last_undegraded":"2024-09-22T03:28:36.891760+0000","last_fullsized":"2024-09-22T03:28:36.891760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.884986+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:36.884986+0000","last_peered":"2024-09-22T03:28:36.884986+0000","last_clean":"2024-09-22T03:28:36.884986+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:36.884986+0000","last_undegraded":"2024-09-22T03:28:36.884986+0000","last_fullsized":"2024-09-22T03:28:36.884986+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.905481+0000","last_change":"2024-09-22T03:28:36.905481+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.905481+0000","last_undegraded":"2024-09-22T03:28:36.905481+0000","last_fullsized":"2024-09-22T03:28:36.905481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.887122+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:36.887122+0000","last_peered":"2024-09-22T03:28:36.887122+0000","last_clean":"2024-09-22T03:28:36.887122+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:36.887122+0000","last_undegraded":"2024-09-22T03:28:36.887122+0000","last_fullsized":"2024-09-22T03:28:36.887122+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214202429440,"internally_reserved":0,"allocated":9064448,"data_stored":9064448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214202429440,"internally_reserved":0,"allocated":9064448,"data_stored":9064448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7272,"kb_used_data":7272,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290456,"statfs":{"total":53552873472,"available":53545426944,"internally_reserved":0,"allocated":7446528,"data_stored":7446528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":9828,"kb_used_data":9828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52287900,"statfs":{"total":53552873472,"available":53542809600,"internally_reserved":0,"allocated":10063872,"data_stored":10063872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2896,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294832,"statfs":{"total":53552873472,"available":53549907968,"internally_reserved":0,"allocated":2965504,"data_stored":2965504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8096,"kb_used_data":8096,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289632,"statfs":{"total":53552873472,"available":53544583168,"internally_reserved":0,"allocated":8290304,"data_stored":8290304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550366720,"internally_reserved":0,"allocated":2506752,"data_stored":2506752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551300608,"internally_reserved":0,"allocated":1572864,"data_stored":1572864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550211072,"internally_reserved":0,"allocated":2662400,"data_stored":2662400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53550366720,"internally_reserved":0,"allocated":2506752,"data_stored":2506752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53551300608,"internally_reserved":0,"allocated":1572864,"data_stored":1572864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53550211072,"internally_reserved":0,"allocated":2662400,"data_stored":2662400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:38.622 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T03:28:38.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.759+0000 7f9fe5b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:38.764+0000 7f9fe5b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:38.897 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-22T03:28:38.899 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:28:38.899 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:28:38.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:28:38.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:28:38.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_44316 2024-09-22T03:28:38.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:28:38.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:28:39.027 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:39.028 INFO:teuthology.orchestra.run.smithi129.stdout:{"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":343,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":60,"num_osds":4,"num_up_osds":4,"osd_up_since":1726975381,"num_in_osds":4,"osd_in_since":1726975377,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":16},{"state_name":"active+clean","count":9}],"num_pgs":25,"num_pools":3,"num_objects":4,"data_bytes":590387,"bytes_used":28766208,"bytes_avail":214182727680,"bytes_total":214211493888,"inactive_pgs_ratio":0.63999998569488525},"fsmap":{"epoch":1,"btime":"2024-09-22T03:22:55:691915+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-22T03:24:01.947109+0000","services":{}},"progress_events":{}} 2024-09-22T03:28:39.044 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-22T03:28:39.044 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:39.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:39.231+0000 7f5b1da2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:39.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:39.238+0000 7f5b1da2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:39.562 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:39.562 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:39.585 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-22T03:28:38.873209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":28092,"kb_used_data":28092,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209162820,"statfs":{"total":214211493888,"available":214182727680,"internally_reserved":0,"allocated":28766208,"data_stored":28766208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26066968,"num_objects":-399,"num_object_clones":0,"num_object_copies":-1197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-16260,"num_write_kb":-118449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.026231"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.938584+0000","last_change":"2024-09-22T03:28:36.938584+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.938584+0000","last_undegraded":"2024-09-22T03:28:36.938584+0000","last_fullsized":"2024-09-22T03:28:36.938584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.932214+0000","last_change":"2024-09-22T03:28:36.932214+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.932214+0000","last_undegraded":"2024-09-22T03:28:36.932214+0000","last_fullsized":"2024-09-22T03:28:36.932214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.917916+0000","last_change":"2024-09-22T03:28:36.917916+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.917916+0000","last_undegraded":"2024-09-22T03:28:36.917916+0000","last_fullsized":"2024-09-22T03:28:36.917916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927289+0000","last_change":"2024-09-22T03:28:36.927289+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927289+0000","last_undegraded":"2024-09-22T03:28:36.927289+0000","last_fullsized":"2024-09-22T03:28:36.927289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.904312+0000","last_change":"2024-09-22T03:28:36.904312+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.904312+0000","last_undegraded":"2024-09-22T03:28:36.904312+0000","last_fullsized":"2024-09-22T03:28:36.904312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927970+0000","last_change":"2024-09-22T03:28:36.927970+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927970+0000","last_undegraded":"2024-09-22T03:28:36.927970+0000","last_fullsized":"2024-09-22T03:28:36.927970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.912291+0000","last_change":"2024-09-22T03:28:36.912291+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.912291+0000","last_undegraded":"2024-09-22T03:28:36.912291+0000","last_fullsized":"2024-09-22T03:28:36.912291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.927974+0000","last_change":"2024-09-22T03:28:36.927974+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.927974+0000","last_undegraded":"2024-09-22T03:28:36.927974+0000","last_fullsized":"2024-09-22T03:28:36.927974+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.916162+0000","last_change":"2024-09-22T03:28:36.916162+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.916162+0000","last_undegraded":"2024-09-22T03:28:36.916162+0000","last_fullsized":"2024-09-22T03:28:36.916162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":277,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.879485+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:36.879485+0000","last_peered":"2024-09-22T03:28:36.879485+0000","last_clean":"2024-09-22T03:28:36.879485+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:36.879485+0000","last_undegraded":"2024-09-22T03:28:36.879485+0000","last_fullsized":"2024-09-22T03:28:36.879485+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":143,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.887714+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:36.887714+0000","last_peered":"2024-09-22T03:28:36.887714+0000","last_clean":"2024-09-22T03:28:36.887714+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:36.887714+0000","last_undegraded":"2024-09-22T03:28:36.887714+0000","last_fullsized":"2024-09-22T03:28:36.887714+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.886782+0000","last_change":"2024-09-22T03:28:36.886782+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.886782+0000","last_undegraded":"2024-09-22T03:28:36.886782+0000","last_fullsized":"2024-09-22T03:28:36.886782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.900203+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:36.900203+0000","last_peered":"2024-09-22T03:28:36.900203+0000","last_clean":"2024-09-22T03:28:36.900203+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:36.900203+0000","last_undegraded":"2024-09-22T03:28:36.900203+0000","last_fullsized":"2024-09-22T03:28:36.900203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.914604+0000","last_change":"2024-09-22T03:28:36.914604+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.914604+0000","last_undegraded":"2024-09-22T03:28:36.914604+0000","last_fullsized":"2024-09-22T03:28:36.914604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.886250+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:36.886250+0000","last_peered":"2024-09-22T03:28:36.886250+0000","last_clean":"2024-09-22T03:28:36.886250+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:36.886250+0000","last_undegraded":"2024-09-22T03:28:36.886250+0000","last_fullsized":"2024-09-22T03:28:36.886250+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.896833+0000","last_change":"2024-09-22T03:28:36.896833+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.896833+0000","last_undegraded":"2024-09-22T03:28:36.896833+0000","last_fullsized":"2024-09-22T03:28:36.896833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":144,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.898290+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:36.898290+0000","last_peered":"2024-09-22T03:28:36.898290+0000","last_clean":"2024-09-22T03:28:36.898290+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:36.898290+0000","last_undegraded":"2024-09-22T03:28:36.898290+0000","last_fullsized":"2024-09-22T03:28:36.898290+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.889697+0000","last_change":"2024-09-22T03:28:36.889697+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.889697+0000","last_undegraded":"2024-09-22T03:28:36.889697+0000","last_fullsized":"2024-09-22T03:28:36.889697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.885663+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:36.885663+0000","last_peered":"2024-09-22T03:28:36.885663+0000","last_clean":"2024-09-22T03:28:36.885663+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:36.885663+0000","last_undegraded":"2024-09-22T03:28:36.885663+0000","last_fullsized":"2024-09-22T03:28:36.885663+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.900131+0000","last_change":"2024-09-22T03:28:36.900131+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.900131+0000","last_undegraded":"2024-09-22T03:28:36.900131+0000","last_fullsized":"2024-09-22T03:28:36.900131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.881697+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:36.881697+0000","last_peered":"2024-09-22T03:28:36.881697+0000","last_clean":"2024-09-22T03:28:36.881697+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:36.881697+0000","last_undegraded":"2024-09-22T03:28:36.881697+0000","last_fullsized":"2024-09-22T03:28:36.881697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.891760+0000","last_change":"2024-09-22T03:28:36.891760+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.891760+0000","last_undegraded":"2024-09-22T03:28:36.891760+0000","last_fullsized":"2024-09-22T03:28:36.891760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.884986+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:36.884986+0000","last_peered":"2024-09-22T03:28:36.884986+0000","last_clean":"2024-09-22T03:28:36.884986+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:36.884986+0000","last_undegraded":"2024-09-22T03:28:36.884986+0000","last_fullsized":"2024-09-22T03:28:36.884986+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-22T03:28:36.905481+0000","last_change":"2024-09-22T03:28:36.905481+0000","last_active":"2024-09-22T03:28:36.857520+0000","last_peered":"2024-09-22T03:28:36.857520+0000","last_clean":"2024-09-22T03:28:36.857520+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:28:36.905481+0000","last_undegraded":"2024-09-22T03:28:36.905481+0000","last_fullsized":"2024-09-22T03:28:36.905481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-22T03:28:36.887122+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:36.887122+0000","last_peered":"2024-09-22T03:28:36.887122+0000","last_clean":"2024-09-22T03:28:36.887122+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:36.887122+0000","last_undegraded":"2024-09-22T03:28:36.887122+0000","last_fullsized":"2024-09-22T03:28:36.887122+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214202429440,"internally_reserved":0,"allocated":9064448,"data_stored":9064448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214202429440,"internally_reserved":0,"allocated":9064448,"data_stored":9064448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7272,"kb_used_data":7272,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290456,"statfs":{"total":53552873472,"available":53545426944,"internally_reserved":0,"allocated":7446528,"data_stored":7446528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":9828,"kb_used_data":9828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52287900,"statfs":{"total":53552873472,"available":53542809600,"internally_reserved":0,"allocated":10063872,"data_stored":10063872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2896,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294832,"statfs":{"total":53552873472,"available":53549907968,"internally_reserved":0,"allocated":2965504,"data_stored":2965504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8096,"kb_used_data":8096,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289632,"statfs":{"total":53552873472,"available":53544583168,"internally_reserved":0,"allocated":8290304,"data_stored":8290304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550366720,"internally_reserved":0,"allocated":2506752,"data_stored":2506752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551300608,"internally_reserved":0,"allocated":1572864,"data_stored":1572864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550211072,"internally_reserved":0,"allocated":2662400,"data_stored":2662400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53550366720,"internally_reserved":0,"allocated":2506752,"data_stored":2506752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53551300608,"internally_reserved":0,"allocated":1572864,"data_stored":1572864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53550211072,"internally_reserved":0,"allocated":2662400,"data_stored":2662400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53550551040,"internally_reserved":0,"allocated":2322432,"data_stored":2322432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:39.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 192 176 1.37491 1.375 0.132978 0.0871645 2024-09-22T03:28:40.417 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:40.547+0000 7f4c9a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:40.554+0000 7f4c9a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:40.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 352 336 1.3124 1.25 0.0296044 0.0908176 2024-09-22T03:28:41.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 510 494 1.28637 1.23438 0.0454365 0.0963728 2024-09-22T03:28:42.588 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:42.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:42.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:42.702+0000 7f77a0703640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:42.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:42.709+0000 7f77a0703640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:42.795+0000 7fc023d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:42.801+0000 7fc023d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:42.926 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:42.926 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:42.940 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-22T03:28:41.997257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4686387,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":639,"num_write_kb":5896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":585,"ondisk_log_size":585,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":28204,"kb_used_data":28204,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209162708,"statfs":{"total":214211493888,"available":214182612992,"internally_reserved":0,"allocated":28880896,"data_stored":28880896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4096000,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500,"num_write_kb":4000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000984"},"pg_stats":[{"pgid":"6.f","version":"60'47","reported_seq":60,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.233398+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:28:41.233398+0000","last_peered":"2024-09-22T03:28:41.233398+0000","last_clean":"2024-09-22T03:28:41.233398+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:28:41.233398+0000","last_undegraded":"2024-09-22T03:28:41.233398+0000","last_fullsized":"2024-09-22T03:28:41.233398+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'47","reported_seq":60,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.794640+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:28:41.794640+0000","last_peered":"2024-09-22T03:28:41.794640+0000","last_clean":"2024-09-22T03:28:41.794640+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:28:41.794640+0000","last_undegraded":"2024-09-22T03:28:41.794640+0000","last_fullsized":"2024-09-22T03:28:41.794640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'31","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.914172+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:28:41.914172+0000","last_peered":"2024-09-22T03:28:41.914172+0000","last_clean":"2024-09-22T03:28:41.914172+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:28:41.914172+0000","last_undegraded":"2024-09-22T03:28:41.914172+0000","last_fullsized":"2024-09-22T03:28:41.914172+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'31","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.179849+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:28:41.179849+0000","last_peered":"2024-09-22T03:28:41.179849+0000","last_clean":"2024-09-22T03:28:41.179849+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:28:41.179849+0000","last_undegraded":"2024-09-22T03:28:41.179849+0000","last_fullsized":"2024-09-22T03:28:41.179849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'39","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.722260+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:28:41.722260+0000","last_peered":"2024-09-22T03:28:41.722260+0000","last_clean":"2024-09-22T03:28:41.722260+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:28:41.722260+0000","last_undegraded":"2024-09-22T03:28:41.722260+0000","last_fullsized":"2024-09-22T03:28:41.722260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.905865+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:28:38.905865+0000","last_peered":"2024-09-22T03:28:38.905865+0000","last_clean":"2024-09-22T03:28:38.905865+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:28:38.905865+0000","last_undegraded":"2024-09-22T03:28:38.905865+0000","last_fullsized":"2024-09-22T03:28:38.905865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'39","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.364558+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:28:41.364558+0000","last_peered":"2024-09-22T03:28:41.364558+0000","last_clean":"2024-09-22T03:28:41.364558+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:28:41.364558+0000","last_undegraded":"2024-09-22T03:28:41.364558+0000","last_fullsized":"2024-09-22T03:28:41.364558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'55","reported_seq":68,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:40.987086+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:28:40.987086+0000","last_peered":"2024-09-22T03:28:40.987086+0000","last_clean":"2024-09-22T03:28:40.987086+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:28:40.987086+0000","last_undegraded":"2024-09-22T03:28:40.987086+0000","last_fullsized":"2024-09-22T03:28:40.987086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'23","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.382175+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:28:41.382175+0000","last_peered":"2024-09-22T03:28:41.382175+0000","last_clean":"2024-09-22T03:28:41.382175+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:28:41.382175+0000","last_undegraded":"2024-09-22T03:28:41.382175+0000","last_fullsized":"2024-09-22T03:28:41.382175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'7","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:40.953534+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:28:40.953534+0000","last_peered":"2024-09-22T03:28:40.953534+0000","last_clean":"2024-09-22T03:28:40.953534+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:28:40.953534+0000","last_undegraded":"2024-09-22T03:28:40.953534+0000","last_fullsized":"2024-09-22T03:28:40.953534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'24","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.874347+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:28:41.874347+0000","last_peered":"2024-09-22T03:28:41.874347+0000","last_clean":"2024-09-22T03:28:41.874347+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:28:41.874347+0000","last_undegraded":"2024-09-22T03:28:41.874347+0000","last_fullsized":"2024-09-22T03:28:41.874347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204800,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'39","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.544601+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:28:41.544601+0000","last_peered":"2024-09-22T03:28:41.544601+0000","last_clean":"2024-09-22T03:28:41.544601+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:28:41.544601+0000","last_undegraded":"2024-09-22T03:28:41.544601+0000","last_fullsized":"2024-09-22T03:28:41.544601+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'10","reported_seq":23,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.962263+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:28:41.962263+0000","last_peered":"2024-09-22T03:28:41.962263+0000","last_clean":"2024-09-22T03:28:41.962263+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:28:41.962263+0000","last_undegraded":"2024-09-22T03:28:41.962263+0000","last_fullsized":"2024-09-22T03:28:41.962263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90112,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'39","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.429921+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:28:41.429921+0000","last_peered":"2024-09-22T03:28:41.429921+0000","last_clean":"2024-09-22T03:28:41.429921+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:28:41.429921+0000","last_undegraded":"2024-09-22T03:28:41.429921+0000","last_fullsized":"2024-09-22T03:28:41.429921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'23","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:41.652893+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:28:41.652893+0000","last_peered":"2024-09-22T03:28:41.652893+0000","last_clean":"2024-09-22T03:28:41.652893+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:28:41.652893+0000","last_undegraded":"2024-09-22T03:28:41.652893+0000","last_fullsized":"2024-09-22T03:28:41.652893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'31","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:40.315413+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:28:40.315413+0000","last_peered":"2024-09-22T03:28:40.315413+0000","last_clean":"2024-09-22T03:28:40.315413+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:28:40.315413+0000","last_undegraded":"2024-09-22T03:28:40.315413+0000","last_fullsized":"2024-09-22T03:28:40.315413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":4096000,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500,"num_write_kb":4000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214175297536,"internally_reserved":0,"allocated":36196352,"data_stored":36196352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":485,"ondisk_log_size":485,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214175297536,"internally_reserved":0,"allocated":36196352,"data_stored":36196352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53544402944,"internally_reserved":0,"allocated":8470528,"data_stored":8470528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6560,"kb_used_data":6560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52291168,"statfs":{"total":53552873472,"available":53546156032,"internally_reserved":0,"allocated":6717440,"data_stored":6717440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7228,"kb_used_data":7228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290500,"statfs":{"total":53552873472,"available":53545472000,"internally_reserved":0,"allocated":7401472,"data_stored":7401472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6716,"kb_used_data":6716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52291012,"statfs":{"total":53552873472,"available":53545996288,"internally_reserved":0,"allocated":6877184,"data_stored":6877184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7700,"kb_used_data":7700,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290028,"statfs":{"total":53552873472,"available":53544988672,"internally_reserved":0,"allocated":7884800,"data_stored":7884800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53544402944,"internally_reserved":0,"allocated":8470528,"data_stored":8470528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53543190528,"internally_reserved":0,"allocated":9682944,"data_stored":9682944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53543931904,"internally_reserved":0,"allocated":8941568,"data_stored":8941568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53543772160,"internally_reserved":0,"allocated":9101312,"data_stored":9101312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53544402944,"internally_reserved":0,"allocated":8470528,"data_stored":8470528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53543190528,"internally_reserved":0,"allocated":9682944,"data_stored":9682944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53543931904,"internally_reserved":0,"allocated":8941568,"data_stored":8941568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53543772160,"internally_reserved":0,"allocated":9101312,"data_stored":9101312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53544402944,"internally_reserved":0,"allocated":8470528,"data_stored":8470528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:42.942 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:28:42.942 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:42.942 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:28:42.942 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:28:42.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 661 645 1.25967 1.17969 0.0431488 0.0962094 2024-09-22T03:28:43.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.084+0000 7fe6ce797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.095+0000 7fe6ce797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:43.111 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:43.112 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:43.112 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:43.112 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:43.115 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:43.115 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:43.116 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:28:43.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.229+0000 7f50ad317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.239+0000 7f50ad317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.273 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:43.273 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:43.273 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:43.273 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:43.273 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:43.273 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:43.273 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:43.274 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:43.274 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:43.274 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:43.274 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:43.274 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:43.277 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:43.278 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:43.278 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:28:43.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.393+0000 7f11f3317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.402+0000 7f11f3317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.425 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:43.425 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:43.425 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:43.425 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:43.426 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:43.428 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:43.429 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:43.429 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:28:43.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.538+0000 7ff5e1f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:43.547+0000 7ff5e1f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:43.561 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:43.561 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:43.561 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:43.562 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:43.565 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:43.565 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:43.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 791 775 1.21085 1.01562 0.150871 0.100702 2024-09-22T03:28:44.917 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:44.955 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 962 946 1.23168 1.33594 0.124381 0.09971 2024-09-22T03:28:45.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:45.052+0000 7fae64c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:45.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:45.061+0000 7fae64c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:45.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 1128 1112 1.24097 1.29688 0.0534002 0.0998645 2024-09-22T03:28:46.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 16 1312 1296 1.26552 1.4375 0.0307421 0.0983693 2024-09-22T03:28:47.208 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:47.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:47.351+0000 7f26db4fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:47.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:47.359+0000 7f26db4fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:47.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1463 1447 1.25598 1.17969 0.0290789 0.0982074 2024-09-22T03:28:48.565 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:48.566 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:48.566 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:48.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:48.701+0000 7ff492258640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:48.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:48.709+0000 7ff492258640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:48.928 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:48.929 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:48.951 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-22T03:28:47.998333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":11297331,"num_objects":168,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1446,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1391,"ondisk_log_size":1391,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":65548,"kb_used_data":65548,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209125364,"statfs":{"total":214211493888,"available":214144372736,"internally_reserved":0,"allocated":67121152,"data_stored":67121152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10706944,"num_objects":164,"num_object_clones":0,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1307,"num_write_kb":10456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001767"},"pg_stats":[{"pgid":"6.f","version":"60'135","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.958772+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:28:46.958772+0000","last_peered":"2024-09-22T03:28:46.958772+0000","last_clean":"2024-09-22T03:28:46.958772+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:28:46.958772+0000","last_undegraded":"2024-09-22T03:28:46.958772+0000","last_fullsized":"2024-09-22T03:28:46.958772+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'71","reported_seq":84,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:45.322293+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:28:45.322293+0000","last_peered":"2024-09-22T03:28:45.322293+0000","last_clean":"2024-09-22T03:28:45.322293+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:28:45.322293+0000","last_undegraded":"2024-09-22T03:28:45.322293+0000","last_fullsized":"2024-09-22T03:28:45.322293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'87","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.673353+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:28:46.673353+0000","last_peered":"2024-09-22T03:28:46.673353+0000","last_clean":"2024-09-22T03:28:46.673353+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:28:46.673353+0000","last_undegraded":"2024-09-22T03:28:46.673353+0000","last_fullsized":"2024-09-22T03:28:46.673353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'63","reported_seq":76,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:45.992526+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:28:45.992526+0000","last_peered":"2024-09-22T03:28:45.992526+0000","last_clean":"2024-09-22T03:28:45.992526+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:28:45.992526+0000","last_undegraded":"2024-09-22T03:28:45.992526+0000","last_fullsized":"2024-09-22T03:28:45.992526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'95","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.870428+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:28:46.870428+0000","last_peered":"2024-09-22T03:28:46.870428+0000","last_clean":"2024-09-22T03:28:46.870428+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:28:46.870428+0000","last_undegraded":"2024-09-22T03:28:46.870428+0000","last_fullsized":"2024-09-22T03:28:46.870428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'66","reported_seq":79,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.965367+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:28:46.965367+0000","last_peered":"2024-09-22T03:28:46.965367+0000","last_clean":"2024-09-22T03:28:46.965367+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:28:46.965367+0000","last_undegraded":"2024-09-22T03:28:46.965367+0000","last_fullsized":"2024-09-22T03:28:46.965367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":548864,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'79","reported_seq":92,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:45.200512+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:28:45.200512+0000","last_peered":"2024-09-22T03:28:45.200512+0000","last_clean":"2024-09-22T03:28:45.200512+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:28:45.200512+0000","last_undegraded":"2024-09-22T03:28:45.200512+0000","last_fullsized":"2024-09-22T03:28:45.200512+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'95","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.737705+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:28:46.737705+0000","last_peered":"2024-09-22T03:28:46.737705+0000","last_clean":"2024-09-22T03:28:46.737705+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:28:46.737705+0000","last_undegraded":"2024-09-22T03:28:46.737705+0000","last_fullsized":"2024-09-22T03:28:46.737705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'71","reported_seq":84,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.603577+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:28:46.603577+0000","last_peered":"2024-09-22T03:28:46.603577+0000","last_clean":"2024-09-22T03:28:46.603577+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:28:46.603577+0000","last_undegraded":"2024-09-22T03:28:46.603577+0000","last_fullsized":"2024-09-22T03:28:46.603577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'87","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.813024+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:28:46.813024+0000","last_peered":"2024-09-22T03:28:46.813024+0000","last_clean":"2024-09-22T03:28:46.813024+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:28:46.813024+0000","last_undegraded":"2024-09-22T03:28:46.813024+0000","last_fullsized":"2024-09-22T03:28:46.813024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'63","reported_seq":76,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.434556+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:28:46.434556+0000","last_peered":"2024-09-22T03:28:46.434556+0000","last_clean":"2024-09-22T03:28:46.434556+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:28:46.434556+0000","last_undegraded":"2024-09-22T03:28:46.434556+0000","last_fullsized":"2024-09-22T03:28:46.434556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'63","reported_seq":76,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.776941+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:28:46.776941+0000","last_peered":"2024-09-22T03:28:46.776941+0000","last_clean":"2024-09-22T03:28:46.776941+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:28:46.776941+0000","last_undegraded":"2024-09-22T03:28:46.776941+0000","last_fullsized":"2024-09-22T03:28:46.776941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'79","reported_seq":92,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.190948+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:28:46.190948+0000","last_peered":"2024-09-22T03:28:46.190948+0000","last_clean":"2024-09-22T03:28:46.190948+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:28:46.190948+0000","last_undegraded":"2024-09-22T03:28:46.190948+0000","last_fullsized":"2024-09-22T03:28:46.190948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'71","reported_seq":84,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:45.241076+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:28:45.241076+0000","last_peered":"2024-09-22T03:28:45.241076+0000","last_clean":"2024-09-22T03:28:45.241076+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:28:45.241076+0000","last_undegraded":"2024-09-22T03:28:45.241076+0000","last_fullsized":"2024-09-22T03:28:45.241076+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'87","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.562215+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:28:46.562215+0000","last_peered":"2024-09-22T03:28:46.562215+0000","last_clean":"2024-09-22T03:28:46.562215+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:28:46.562215+0000","last_undegraded":"2024-09-22T03:28:46.562215+0000","last_fullsized":"2024-09-22T03:28:46.562215+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'79","reported_seq":92,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:46.134249+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:28:46.134249+0000","last_peered":"2024-09-22T03:28:46.134249+0000","last_clean":"2024-09-22T03:28:46.134249+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:28:46.134249+0000","last_undegraded":"2024-09-22T03:28:46.134249+0000","last_fullsized":"2024-09-22T03:28:46.134249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":10706944,"num_objects":164,"num_object_clones":0,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1307,"num_write_kb":10456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214135525376,"internally_reserved":0,"allocated":75968512,"data_stored":75968512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1291,"ondisk_log_size":1291,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214135541760,"internally_reserved":0,"allocated":75952128,"data_stored":75952128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53535793152,"internally_reserved":0,"allocated":17080320,"data_stored":17080320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":14608,"kb_used_data":14608,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52283120,"statfs":{"total":53552873472,"available":53537914880,"internally_reserved":0,"allocated":14958592,"data_stored":14958592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":15896,"kb_used_data":15896,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52281832,"statfs":{"total":53552873472,"available":53536595968,"internally_reserved":0,"allocated":16277504,"data_stored":16277504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":17084,"kb_used_data":17084,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52280644,"statfs":{"total":53552873472,"available":53535379456,"internally_reserved":0,"allocated":17494016,"data_stored":17494016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":17960,"kb_used_data":17960,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52279768,"statfs":{"total":53552873472,"available":53534482432,"internally_reserved":0,"allocated":18391040,"data_stored":18391040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53535793152,"internally_reserved":0,"allocated":17080320,"data_stored":17080320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53532037120,"internally_reserved":0,"allocated":20836352,"data_stored":20836352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53533028352,"internally_reserved":0,"allocated":19845120,"data_stored":19845120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53534683136,"internally_reserved":0,"allocated":18190336,"data_stored":18190336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53535793152,"internally_reserved":0,"allocated":17080320,"data_stored":17080320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53532028928,"internally_reserved":0,"allocated":20844544,"data_stored":20844544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53533020160,"internally_reserved":0,"allocated":19853312,"data_stored":19853312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53534683136,"internally_reserved":0,"allocated":18190336,"data_stored":18190336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53535793152,"internally_reserved":0,"allocated":17080320,"data_stored":17080320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:48.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 1635 1619 1.26474 1.34375 0.0470576 0.0983606 2024-09-22T03:28:49.506 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:49.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:49.638+0000 7f4ca6efa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:49.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:49.644+0000 7f4ca6efa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:49.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 1795 1779 1.26339 1.25 0.0439533 0.0985465 2024-09-22T03:28:50.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 1962 1946 1.26682 1.30469 0.0295375 0.0981795 2024-09-22T03:28:51.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:51.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:51.920+0000 7f51010ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:51.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:51.926+0000 7f51010ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:51.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 2145 2129 1.27934 1.42969 0.0618076 0.0974134 2024-09-22T03:28:52.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 2297 2281 1.27277 1.1875 0.122994 0.0975927 2024-09-22T03:28:53.953 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:53.953 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:53.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:28:53.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 2443 2427 1.26396 1.14062 0.119893 0.0983826 2024-09-22T03:28:54.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:54.083+0000 7fc9a0557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:54.093+0000 7fc9a0557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.098 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:54.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:54.225+0000 7f7a1106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:54.231+0000 7f7a1106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:28:54.351 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:28:54.368 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-22T03:28:53.999512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18170419,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2285,"num_write_kb":19064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2230,"ondisk_log_size":2230,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":105912,"kb_used_data":105912,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209085000,"statfs":{"total":214211493888,"available":214103040000,"internally_reserved":0,"allocated":108453888,"data_stored":108453888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13484032,"num_objects":205,"num_object_clones":0,"num_object_copies":615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1646,"num_write_kb":13168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002254"},"pg_stats":[{"pgid":"6.f","version":"60'193","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.960476+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:28:51.960476+0000","last_peered":"2024-09-22T03:28:51.960476+0000","last_clean":"2024-09-22T03:28:51.960476+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:28:51.960476+0000","last_undegraded":"2024-09-22T03:28:51.960476+0000","last_fullsized":"2024-09-22T03:28:51.960476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589248,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'103","reported_seq":116,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.123028+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:28:51.123028+0000","last_peered":"2024-09-22T03:28:51.123028+0000","last_clean":"2024-09-22T03:28:51.123028+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:28:51.123028+0000","last_undegraded":"2024-09-22T03:28:51.123028+0000","last_fullsized":"2024-09-22T03:28:51.123028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'159","reported_seq":172,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.713458+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:28:51.713458+0000","last_peered":"2024-09-22T03:28:51.713458+0000","last_clean":"2024-09-22T03:28:51.713458+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:28:51.713458+0000","last_undegraded":"2024-09-22T03:28:51.713458+0000","last_fullsized":"2024-09-22T03:28:51.713458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'119","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.947124+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:28:51.947124+0000","last_peered":"2024-09-22T03:28:51.947124+0000","last_clean":"2024-09-22T03:28:51.947124+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:28:51.947124+0000","last_undegraded":"2024-09-22T03:28:51.947124+0000","last_fullsized":"2024-09-22T03:28:51.947124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'135","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.235433+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:28:51.235433+0000","last_peered":"2024-09-22T03:28:51.235433+0000","last_clean":"2024-09-22T03:28:51.235433+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:28:51.235433+0000","last_undegraded":"2024-09-22T03:28:51.235433+0000","last_fullsized":"2024-09-22T03:28:51.235433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'143","reported_seq":156,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.077460+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:28:51.077460+0000","last_peered":"2024-09-22T03:28:51.077460+0000","last_clean":"2024-09-22T03:28:51.077460+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:28:51.077460+0000","last_undegraded":"2024-09-22T03:28:51.077460+0000","last_fullsized":"2024-09-22T03:28:51.077460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'127","reported_seq":140,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.641295+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:28:51.641295+0000","last_peered":"2024-09-22T03:28:51.641295+0000","last_clean":"2024-09-22T03:28:51.641295+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:28:51.641295+0000","last_undegraded":"2024-09-22T03:28:51.641295+0000","last_fullsized":"2024-09-22T03:28:51.641295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'119","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:49.399598+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:28:49.399598+0000","last_peered":"2024-09-22T03:28:49.399598+0000","last_clean":"2024-09-22T03:28:49.399598+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:28:49.399598+0000","last_undegraded":"2024-09-22T03:28:49.399598+0000","last_fullsized":"2024-09-22T03:28:49.399598+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'127","reported_seq":140,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.833285+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:28:51.833285+0000","last_peered":"2024-09-22T03:28:51.833285+0000","last_clean":"2024-09-22T03:28:51.833285+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:28:51.833285+0000","last_undegraded":"2024-09-22T03:28:51.833285+0000","last_fullsized":"2024-09-22T03:28:51.833285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'159","reported_seq":172,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:49.627440+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:28:49.627440+0000","last_peered":"2024-09-22T03:28:49.627440+0000","last_clean":"2024-09-22T03:28:49.627440+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:28:49.627440+0000","last_undegraded":"2024-09-22T03:28:49.627440+0000","last_fullsized":"2024-09-22T03:28:49.627440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'135","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.829911+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:28:51.829911+0000","last_peered":"2024-09-22T03:28:51.829911+0000","last_clean":"2024-09-22T03:28:51.829911+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:28:51.829911+0000","last_undegraded":"2024-09-22T03:28:51.829911+0000","last_fullsized":"2024-09-22T03:28:51.829911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'103","reported_seq":116,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.443454+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:28:51.443454+0000","last_peered":"2024-09-22T03:28:51.443454+0000","last_clean":"2024-09-22T03:28:51.443454+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:28:51.443454+0000","last_undegraded":"2024-09-22T03:28:51.443454+0000","last_fullsized":"2024-09-22T03:28:51.443454+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'135","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.187761+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:28:51.187761+0000","last_peered":"2024-09-22T03:28:51.187761+0000","last_clean":"2024-09-22T03:28:51.187761+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:28:51.187761+0000","last_undegraded":"2024-09-22T03:28:51.187761+0000","last_fullsized":"2024-09-22T03:28:51.187761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'111","reported_seq":124,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.575811+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:28:51.575811+0000","last_peered":"2024-09-22T03:28:51.575811+0000","last_clean":"2024-09-22T03:28:51.575811+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:28:51.575811+0000","last_undegraded":"2024-09-22T03:28:51.575811+0000","last_fullsized":"2024-09-22T03:28:51.575811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'151","reported_seq":164,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.920245+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:28:51.920245+0000","last_peered":"2024-09-22T03:28:51.920245+0000","last_clean":"2024-09-22T03:28:51.920245+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:28:51.920245+0000","last_undegraded":"2024-09-22T03:28:51.920245+0000","last_fullsized":"2024-09-22T03:28:51.920245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'111","reported_seq":124,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:51.407653+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:28:51.407653+0000","last_peered":"2024-09-22T03:28:51.407653+0000","last_clean":"2024-09-22T03:28:51.407653+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:28:51.407653+0000","last_undegraded":"2024-09-22T03:28:51.407653+0000","last_fullsized":"2024-09-22T03:28:51.407653+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":17580032,"num_objects":269,"num_object_clones":0,"num_object_copies":807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2146,"num_write_kb":17168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214094286848,"internally_reserved":0,"allocated":117207040,"data_stored":117207040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2130,"ondisk_log_size":2130,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214094286848,"internally_reserved":0,"allocated":117207040,"data_stored":117207040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53526581248,"internally_reserved":0,"allocated":26292224,"data_stored":26292224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23640,"kb_used_data":23640,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274088,"statfs":{"total":53552873472,"available":53528666112,"internally_reserved":0,"allocated":24207360,"data_stored":24207360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":25056,"kb_used_data":25056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52272672,"statfs":{"total":53552873472,"available":53527216128,"internally_reserved":0,"allocated":25657344,"data_stored":25657344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":28028,"kb_used_data":28028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52269700,"statfs":{"total":53552873472,"available":53524172800,"internally_reserved":0,"allocated":28700672,"data_stored":28700672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":29188,"kb_used_data":29188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52268540,"statfs":{"total":53552873472,"available":53522984960,"internally_reserved":0,"allocated":29888512,"data_stored":29888512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53526581248,"internally_reserved":0,"allocated":26292224,"data_stored":26292224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53520572416,"internally_reserved":0,"allocated":32301056,"data_stored":32301056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53521793024,"internally_reserved":0,"allocated":31080448,"data_stored":31080448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53525348352,"internally_reserved":0,"allocated":27525120,"data_stored":27525120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53526573056,"internally_reserved":0,"allocated":26300416,"data_stored":26300416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53520572416,"internally_reserved":0,"allocated":32301056,"data_stored":32301056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53521793024,"internally_reserved":0,"allocated":31080448,"data_stored":31080448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53525348352,"internally_reserved":0,"allocated":27525120,"data_stored":27525120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53526573056,"internally_reserved":0,"allocated":26300416,"data_stored":26300416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:28:54.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 2606 2590 1.26454 1.27344 0.174306 0.0981896 2024-09-22T03:28:55.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 2756 2740 1.25909 1.17188 0.0829177 0.0988538 2024-09-22T03:28:56.376 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:28:56.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:56.478+0000 7fb668c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:56.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:56.483+0000 7fb668c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:56.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 2911 2895 1.2564 1.21094 0.0967948 0.0991578 2024-09-22T03:28:57.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 3063 3047 1.25277 1.1875 0.132126 0.0992794 2024-09-22T03:28:58.628 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:28:58.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:58.736+0000 7f5dbb597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:58.741+0000 7f5dbb597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:58.956 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:28:58.956541+0000 min lat: 0.0127648 max lat: 0.453867 avg lat: 0.100499 2024-09-22T03:28:58.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:28:58.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 3186 3170 1.23818 0.960938 0.0911856 0.100499 2024-09-22T03:28:59.370 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:28:59.370 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:28:59.371 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:28:59.371 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:28:59.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.482+0000 7fd1f282c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.491+0000 7fd1f282c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:59.517 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:59.520 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:59.521 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:59.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:28:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.646+0000 7fc3e2cfa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.655+0000 7fc3e2cfa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.672 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:59.672 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:59.673 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:59.676 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:59.676 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:59.676 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:28:59.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.794+0000 7fd0e1e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.805+0000 7fd0e1e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:28:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:28:59.846 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:28:59.846 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:28:59.846 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:28:59.846 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:28:59.849 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:28:59.849 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:28:59.849 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:28:59.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 3338 3322 1.23576 1.1875 0.0706162 0.100749 2024-09-22T03:28:59.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.988+0000 7fab4a90e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:00.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:28:59.999+0000 7fab4a90e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:00.019 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:00.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:00.023 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:00.925 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:00.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 3519 3503 1.24386 1.41406 0.0751901 0.100262 2024-09-22T03:29:01.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:01.053+0000 7f89c2d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:01.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:01.059+0000 7f89c2d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:01.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 3685 3669 1.24616 1.29688 0.0750815 0.0999335 2024-09-22T03:29:02.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 3835 3819 1.24306 1.17188 0.251015 0.100133 2024-09-22T03:29:03.242 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:03.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:03.381+0000 7f9bcf263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:03.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:03.387+0000 7f9bcf263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:03.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 3995 3979 1.24333 1.25 0.166105 0.100251 2024-09-22T03:29:04.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 4148 4132 1.24148 1.19531 0.139811 0.100388 2024-09-22T03:29:05.024 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:05.024 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:05.024 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:05.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:05.152+0000 7f196964f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:05.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:05.161+0000 7f196964f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:05.400 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:29:05.400 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:29:05.416 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-22T03:29:04.001477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30761523,"num_objects":465,"num_object_clones":0,"num_object_copies":1395,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3822,"num_write_kb":31360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3767,"ondisk_log_size":3767,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":182492,"kb_used_data":182492,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209008420,"statfs":{"total":214211493888,"available":214024622080,"internally_reserved":0,"allocated":186871808,"data_stored":186871808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12591104,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1537,"num_write_kb":12296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002345"},"pg_stats":[{"pgid":"6.f","version":"60'247","reported_seq":260,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:58.519699+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:28:58.519699+0000","last_peered":"2024-09-22T03:28:58.519699+0000","last_clean":"2024-09-22T03:28:58.519699+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:28:58.519699+0000","last_undegraded":"2024-09-22T03:28:58.519699+0000","last_fullsized":"2024-09-22T03:28:58.519699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'231","reported_seq":244,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.181992+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:29:01.181992+0000","last_peered":"2024-09-22T03:29:01.181992+0000","last_clean":"2024-09-22T03:29:01.181992+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:29:01.181992+0000","last_undegraded":"2024-09-22T03:29:01.181992+0000","last_fullsized":"2024-09-22T03:29:01.181992+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'255","reported_seq":268,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.938317+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:29:01.938317+0000","last_peered":"2024-09-22T03:29:01.938317+0000","last_clean":"2024-09-22T03:29:01.938317+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:29:01.938317+0000","last_undegraded":"2024-09-22T03:29:01.938317+0000","last_fullsized":"2024-09-22T03:29:01.938317+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'231","reported_seq":244,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.534251+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:29:01.534251+0000","last_peered":"2024-09-22T03:29:01.534251+0000","last_clean":"2024-09-22T03:29:01.534251+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:29:01.534251+0000","last_undegraded":"2024-09-22T03:29:01.534251+0000","last_fullsized":"2024-09-22T03:29:01.534251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'207","reported_seq":220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.380991+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:29:00.380991+0000","last_peered":"2024-09-22T03:29:00.380991+0000","last_clean":"2024-09-22T03:29:00.380991+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:29:00.380991+0000","last_undegraded":"2024-09-22T03:29:00.380991+0000","last_fullsized":"2024-09-22T03:29:00.380991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'247","reported_seq":260,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.850567+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:29:01.850567+0000","last_peered":"2024-09-22T03:29:01.850567+0000","last_clean":"2024-09-22T03:29:01.850567+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:29:01.850567+0000","last_undegraded":"2024-09-22T03:29:01.850567+0000","last_fullsized":"2024-09-22T03:29:01.850567+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'215","reported_seq":228,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.803295+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:29:00.803295+0000","last_peered":"2024-09-22T03:29:00.803295+0000","last_clean":"2024-09-22T03:29:00.803295+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:29:00.803295+0000","last_undegraded":"2024-09-22T03:29:00.803295+0000","last_fullsized":"2024-09-22T03:29:00.803295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'199","reported_seq":212,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.812968+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:29:01.812968+0000","last_peered":"2024-09-22T03:29:01.812968+0000","last_clean":"2024-09-22T03:29:01.812968+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:29:01.812968+0000","last_undegraded":"2024-09-22T03:29:01.812968+0000","last_fullsized":"2024-09-22T03:29:01.812968+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'247","reported_seq":260,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.601498+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:29:01.601498+0000","last_peered":"2024-09-22T03:29:01.601498+0000","last_clean":"2024-09-22T03:29:01.601498+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:29:01.601498+0000","last_undegraded":"2024-09-22T03:29:01.601498+0000","last_fullsized":"2024-09-22T03:29:01.601498+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'263","reported_seq":276,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.766785+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:29:01.766785+0000","last_peered":"2024-09-22T03:29:01.766785+0000","last_clean":"2024-09-22T03:29:01.766785+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:29:01.766785+0000","last_undegraded":"2024-09-22T03:29:01.766785+0000","last_fullsized":"2024-09-22T03:29:01.766785+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'239","reported_seq":252,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.318751+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:29:00.318751+0000","last_peered":"2024-09-22T03:29:00.318751+0000","last_clean":"2024-09-22T03:29:00.318751+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:29:00.318751+0000","last_undegraded":"2024-09-22T03:29:00.318751+0000","last_fullsized":"2024-09-22T03:29:00.318751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'207","reported_seq":220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.411792+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:29:01.411792+0000","last_peered":"2024-09-22T03:29:01.411792+0000","last_clean":"2024-09-22T03:29:01.411792+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:29:01.411792+0000","last_undegraded":"2024-09-22T03:29:01.411792+0000","last_fullsized":"2024-09-22T03:29:01.411792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'215","reported_seq":228,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:00.160350+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:29:00.160350+0000","last_peered":"2024-09-22T03:29:00.160350+0000","last_clean":"2024-09-22T03:29:00.160350+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:29:00.160350+0000","last_undegraded":"2024-09-22T03:29:00.160350+0000","last_fullsized":"2024-09-22T03:29:00.160350+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'207","reported_seq":220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.194617+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:29:01.194617+0000","last_peered":"2024-09-22T03:29:01.194617+0000","last_clean":"2024-09-22T03:29:01.194617+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:29:01.194617+0000","last_undegraded":"2024-09-22T03:29:01.194617+0000","last_fullsized":"2024-09-22T03:29:01.194617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'263","reported_seq":276,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:59.329949+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:28:59.329949+0000","last_peered":"2024-09-22T03:28:59.329949+0000","last_clean":"2024-09-22T03:28:59.329949+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:28:59.329949+0000","last_undegraded":"2024-09-22T03:28:59.329949+0000","last_fullsized":"2024-09-22T03:28:59.329949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'194","reported_seq":207,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:01.960326+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:29:01.960326+0000","last_peered":"2024-09-22T03:29:01.960326+0000","last_clean":"2024-09-22T03:29:01.960326+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:29:01.960326+0000","last_undegraded":"2024-09-22T03:29:01.960326+0000","last_fullsized":"2024-09-22T03:29:01.960326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1597440,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":195,"num_write_kb":1560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":30171136,"num_objects":461,"num_object_clones":0,"num_object_copies":1383,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3683,"num_write_kb":29464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214016221184,"internally_reserved":0,"allocated":195272704,"data_stored":195272704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3667,"ondisk_log_size":3667,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214016237568,"internally_reserved":0,"allocated":195256320,"data_stored":195256320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53508902912,"internally_reserved":0,"allocated":43970560,"data_stored":43970560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":40816,"kb_used_data":40816,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52256912,"statfs":{"total":53552873472,"available":53511077888,"internally_reserved":0,"allocated":41795584,"data_stored":41795584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":42644,"kb_used_data":42644,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52255084,"statfs":{"total":53552873472,"available":53509206016,"internally_reserved":0,"allocated":43667456,"data_stored":43667456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":49036,"kb_used_data":49036,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52248692,"statfs":{"total":53552873472,"available":53502660608,"internally_reserved":0,"allocated":50212864,"data_stored":50212864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":49996,"kb_used_data":49996,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52247732,"statfs":{"total":53552873472,"available":53501677568,"internally_reserved":0,"allocated":51195904,"data_stored":51195904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53508902912,"internally_reserved":0,"allocated":43970560,"data_stored":43970560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53499273216,"internally_reserved":0,"allocated":53600256,"data_stored":53600256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53500145664,"internally_reserved":0,"allocated":52727808,"data_stored":52727808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53507915776,"internally_reserved":0,"allocated":44957696,"data_stored":44957696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53508902912,"internally_reserved":0,"allocated":43970560,"data_stored":43970560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53499265024,"internally_reserved":0,"allocated":53608448,"data_stored":53608448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53500137472,"internally_reserved":0,"allocated":52736000,"data_stored":52736000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53507915776,"internally_reserved":0,"allocated":44957696,"data_stored":44957696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53508902912,"internally_reserved":0,"allocated":43970560,"data_stored":43970560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:05.543 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:05.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:05.656+0000 7f0ef0917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:05.662+0000 7f0ef0917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:05.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 4298 4282 1.2389 1.17188 0.0787977 0.100685 2024-09-22T03:29:06.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 4465 4449 1.24124 1.30469 0.0616459 0.100552 2024-09-22T03:29:07.855 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:07.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 4625 4609 1.24154 1.25 0.0852838 0.100516 2024-09-22T03:29:08.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:07.999+0000 7ffb5b623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:08.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:08.004+0000 7ffb5b623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:08.957 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 4787 4771 1.24234 1.26562 0.0632699 0.100474 2024-09-22T03:29:09.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 4962 4946 1.24636 1.36719 0.041309 0.100125 2024-09-22T03:29:10.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:10.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.315+0000 7fc95f157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.321+0000 7fc95f157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.418 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:10.418 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:10.418 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:29:10.419 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:29:10.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.559+0000 7f6df542c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.569+0000 7f6df542c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.597 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:10.597 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:10.598 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:10.600 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:10.601 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:10.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:29:10.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.718+0000 7f87908ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.729+0000 7f87908ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:10.760 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:10.761 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:10.761 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:10.761 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:10.761 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:10.762 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:10.763 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:10.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:29:10.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.924+0000 7f5d4ef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:10.936+0000 7f5d4ef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:10.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 16 5119 5103 1.24574 1.22656 0.0582757 0.10021 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:10.980 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:10.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:10.984 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:10.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:29:11.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:11.105+0000 7f0c09f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:11.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:11.116+0000 7f0c09f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:11.141 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:11.141 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:11.141 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:11.142 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:11.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:11.145 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:11.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 5280 5264 1.2461 1.25781 0.254901 0.100119 2024-09-22T03:29:12.511 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:12.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:12.666+0000 7ff64f397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:12.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:12.672+0000 7ff64f397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:12.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 5433 5417 1.2446 1.19531 0.0582146 0.10011 2024-09-22T03:29:13.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 5597 5581 1.24565 1.28125 0.0850217 0.100175 2024-09-22T03:29:14.809 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:14.935+0000 7f76bf157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:14.942+0000 7f76bf157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:14.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 5751 5735 1.24446 1.20312 0.031964 0.100304 2024-09-22T03:29:15.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 5905 5889 1.24334 1.20312 0.0994613 0.100309 2024-09-22T03:29:16.145 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:16.146 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:16.146 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:29:16.146 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:29:16.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.305+0000 7f7acef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.317+0000 7f7acef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:16.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:16.350 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:16.350 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:16.353 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:16.353 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:16.354 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:29:16.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.478+0000 7f7d20797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.488+0000 7f7d20797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:16.505 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:16.506 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:16.506 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:16.506 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:16.506 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:16.506 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:16.508 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:16.509 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:16.509 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:29:16.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.633+0000 7fc18e0c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.642+0000 7fc18e0c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:16.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:16.659 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:16.661 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:16.661 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:16.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:29:16.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.795+0000 7f5a6a46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:16.805+0000 7f5a6a46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:16.825 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:16.828 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:16.828 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:16.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 6084 6068 1.24742 1.39844 0.059715 0.100053 2024-09-22T03:29:17.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:17.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:17.284+0000 7f7760e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:17.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:17.289+0000 7f7760e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:17.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 6237 6221 1.24608 1.19531 0.264777 0.100184 2024-09-22T03:29:18.958 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:29:18.958365+0000 min lat: 0.0127648 max lat: 0.472956 avg lat: 0.100292 2024-09-22T03:29:18.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:29:18.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 6385 6369 1.24384 1.15625 0.193229 0.100292 2024-09-22T03:29:19.515 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:19.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:19.670+0000 7f88f970e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:19.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:19.675+0000 7f88f970e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:19.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 6529 6513 1.24093 1.125 0.0961294 0.100622 2024-09-22T03:29:20.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 6683 6667 1.24003 1.20312 0.0710143 0.100666 2024-09-22T03:29:21.829 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:21.829 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:21.829 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:29:21.830 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:29:21.843 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:21.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:21.948+0000 7f0c1d317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:21.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 16 6840 6824 1.23972 1.22656 0.115611 0.100664 2024-09-22T03:29:21.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:21.960+0000 7f0c1d317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:21.977 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:21.977 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:21.977 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:21.977 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:21.977 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:21.977 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:21.978 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:21.978 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:21.978 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:21.978 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:21.978 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:21.978 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:21.982 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:21.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:29:22.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:21.999+0000 7fb76ff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:22.005+0000 7fb76ff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:22.094+0000 7f507b917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:22.104+0000 7f507b917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:22.191 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:22.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:22.195 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:22.195 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:29:22.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:22.344+0000 7fe1ee597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:22.354+0000 7fe1ee597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:22.373 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:22.374 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:22.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:22.376 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:22.376 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:29:22.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:22.496+0000 7f5d86b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:22.509+0000 7f5d86b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:22.531 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:22.535 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:22.535 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:22.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 6973 6957 1.23515 1.03906 0.0347348 0.101006 2024-09-22T03:29:23.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 7141 7125 1.23687 1.3125 0.0617139 0.100939 2024-09-22T03:29:24.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:24.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:24.272+0000 7fd161f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:24.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:24.278+0000 7fd161f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:24.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 7286 7270 1.23461 1.13281 0.0448763 0.101035 2024-09-22T03:29:25.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 16 7457 7441 1.23676 1.33594 0.0680435 0.100923 2024-09-22T03:29:26.379 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:26.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:26.496+0000 7fe943e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:26.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:26.503+0000 7fe943e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:26.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 7607 7591 1.23541 1.17188 0.0976578 0.101033 2024-09-22T03:29:27.536 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:27.537 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:27.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:27.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:27.657+0000 7f30642ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:27.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:27.667+0000 7f30642ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:27.888 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:29:27.888 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:29:27.906 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-22T03:29:26.005572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":56599091,"num_objects":859,"num_object_clones":0,"num_object_copies":2577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6976,"num_write_kb":56592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6921,"ondisk_log_size":6921,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":341320,"kb_used_data":341320,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208849592,"statfs":{"total":214211493888,"available":213861982208,"internally_reserved":0,"allocated":349511680,"data_stored":349511680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12746752,"num_objects":194,"num_object_clones":0,"num_object_copies":582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002265"},"pg_stats":[{"pgid":"6.f","version":"60'431","reported_seq":444,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.142943+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:29:21.142943+0000","last_peered":"2024-09-22T03:29:21.142943+0000","last_clean":"2024-09-22T03:29:21.142943+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:29:21.142943+0000","last_undegraded":"2024-09-22T03:29:21.142943+0000","last_fullsized":"2024-09-22T03:29:21.142943+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'375","reported_seq":388,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.947310+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:29:21.947310+0000","last_peered":"2024-09-22T03:29:21.947310+0000","last_clean":"2024-09-22T03:29:21.947310+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:29:21.947310+0000","last_undegraded":"2024-09-22T03:29:21.947310+0000","last_fullsized":"2024-09-22T03:29:21.947310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'415","reported_seq":428,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.356812+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:29:21.356812+0000","last_peered":"2024-09-22T03:29:21.356812+0000","last_clean":"2024-09-22T03:29:21.356812+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:29:21.356812+0000","last_undegraded":"2024-09-22T03:29:21.356812+0000","last_fullsized":"2024-09-22T03:29:21.356812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'471","reported_seq":484,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.640713+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:29:21.640713+0000","last_peered":"2024-09-22T03:29:21.640713+0000","last_clean":"2024-09-22T03:29:21.640713+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:29:21.640713+0000","last_undegraded":"2024-09-22T03:29:21.640713+0000","last_fullsized":"2024-09-22T03:29:21.640713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'383","reported_seq":396,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:19.782842+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:29:19.782842+0000","last_peered":"2024-09-22T03:29:19.782842+0000","last_clean":"2024-09-22T03:29:19.782842+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:29:19.782842+0000","last_undegraded":"2024-09-22T03:29:19.782842+0000","last_fullsized":"2024-09-22T03:29:19.782842+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'487","reported_seq":500,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.773662+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:29:21.773662+0000","last_peered":"2024-09-22T03:29:21.773662+0000","last_clean":"2024-09-22T03:29:21.773662+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:29:21.773662+0000","last_undegraded":"2024-09-22T03:29:21.773662+0000","last_fullsized":"2024-09-22T03:29:21.773662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'399","reported_seq":412,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.203347+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:29:21.203347+0000","last_peered":"2024-09-22T03:29:21.203347+0000","last_clean":"2024-09-22T03:29:21.203347+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:29:21.203347+0000","last_undegraded":"2024-09-22T03:29:21.203347+0000","last_fullsized":"2024-09-22T03:29:21.203347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'343","reported_seq":356,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.583283+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:29:21.583283+0000","last_peered":"2024-09-22T03:29:21.583283+0000","last_clean":"2024-09-22T03:29:21.583283+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:29:21.583283+0000","last_undegraded":"2024-09-22T03:29:21.583283+0000","last_fullsized":"2024-09-22T03:29:21.583283+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'431","reported_seq":444,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.420862+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:29:21.420862+0000","last_peered":"2024-09-22T03:29:21.420862+0000","last_clean":"2024-09-22T03:29:21.420862+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:29:21.420862+0000","last_undegraded":"2024-09-22T03:29:21.420862+0000","last_fullsized":"2024-09-22T03:29:21.420862+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'487","reported_seq":500,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.691186+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:29:21.691186+0000","last_peered":"2024-09-22T03:29:21.691186+0000","last_clean":"2024-09-22T03:29:21.691186+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:29:21.691186+0000","last_undegraded":"2024-09-22T03:29:21.691186+0000","last_fullsized":"2024-09-22T03:29:21.691186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'492","reported_seq":505,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.962891+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:29:21.962891+0000","last_peered":"2024-09-22T03:29:21.962891+0000","last_clean":"2024-09-22T03:29:21.962891+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:29:21.962891+0000","last_undegraded":"2024-09-22T03:29:21.962891+0000","last_fullsized":"2024-09-22T03:29:21.962891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4038656,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":493,"num_write_kb":3944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'391","reported_seq":404,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.609816+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:29:21.609816+0000","last_peered":"2024-09-22T03:29:21.609816+0000","last_clean":"2024-09-22T03:29:21.609816+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:29:21.609816+0000","last_undegraded":"2024-09-22T03:29:21.609816+0000","last_fullsized":"2024-09-22T03:29:21.609816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'407","reported_seq":420,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.872730+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:29:21.872730+0000","last_peered":"2024-09-22T03:29:21.872730+0000","last_clean":"2024-09-22T03:29:21.872730+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:29:21.872730+0000","last_undegraded":"2024-09-22T03:29:21.872730+0000","last_fullsized":"2024-09-22T03:29:21.872730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'399","reported_seq":412,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.120996+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:29:21.120996+0000","last_peered":"2024-09-22T03:29:21.120996+0000","last_clean":"2024-09-22T03:29:21.120996+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:29:21.120996+0000","last_undegraded":"2024-09-22T03:29:21.120996+0000","last_fullsized":"2024-09-22T03:29:21.120996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'455","reported_seq":468,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:19.986174+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:29:19.986174+0000","last_peered":"2024-09-22T03:29:19.986174+0000","last_clean":"2024-09-22T03:29:19.986174+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:29:19.986174+0000","last_undegraded":"2024-09-22T03:29:19.986174+0000","last_fullsized":"2024-09-22T03:29:19.986174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'455","reported_seq":468,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:21.299558+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:29:21.299558+0000","last_peered":"2024-09-22T03:29:21.299558+0000","last_clean":"2024-09-22T03:29:21.299558+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:29:21.299558+0000","last_undegraded":"2024-09-22T03:29:21.299558+0000","last_fullsized":"2024-09-22T03:29:21.299558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":56008704,"num_objects":855,"num_object_clones":0,"num_object_copies":2565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6837,"num_write_kb":54696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213854576640,"internally_reserved":0,"allocated":356917248,"data_stored":356917248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6821,"ondisk_log_size":6821,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213854584832,"internally_reserved":0,"allocated":356909056,"data_stored":356909056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53471191040,"internally_reserved":0,"allocated":81682432,"data_stored":81682432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":78028,"kb_used_data":78028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52219700,"statfs":{"total":53552873472,"available":53472972800,"internally_reserved":0,"allocated":79900672,"data_stored":79900672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":78044,"kb_used_data":78044,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52219684,"statfs":{"total":53552873472,"available":53472956416,"internally_reserved":0,"allocated":79917056,"data_stored":79917056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":92344,"kb_used_data":92344,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52205384,"statfs":{"total":53552873472,"available":53458313216,"internally_reserved":0,"allocated":94560256,"data_stored":94560256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":92904,"kb_used_data":92904,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52204824,"statfs":{"total":53552873472,"available":53457739776,"internally_reserved":0,"allocated":95133696,"data_stored":95133696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53471191040,"internally_reserved":0,"allocated":81682432,"data_stored":81682432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53455732736,"internally_reserved":0,"allocated":97140736,"data_stored":97140736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53456269312,"internally_reserved":0,"allocated":96604160,"data_stored":96604160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53471391744,"internally_reserved":0,"allocated":81481728,"data_stored":81481728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53471191040,"internally_reserved":0,"allocated":81682432,"data_stored":81682432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53455732736,"internally_reserved":0,"allocated":97140736,"data_stored":97140736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53456261120,"internally_reserved":0,"allocated":96612352,"data_stored":96612352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53471391744,"internally_reserved":0,"allocated":81481728,"data_stored":81481728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53471191040,"internally_reserved":0,"allocated":81682432,"data_stored":81682432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:27.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 7753 7737 1.23347 1.14062 0.112823 0.101191 2024-09-22T03:29:28.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:28.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:28.818+0000 7fbd04f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:28.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:28.826+0000 7fbd04f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:28.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 7891 7875 1.23036 1.07812 0.0948445 0.101491 2024-09-22T03:29:29.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 8058 8042 1.23182 1.30469 0.0748515 0.101381 2024-09-22T03:29:30.959 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 8217 8201 1.23201 1.24219 0.0831277 0.101341 2024-09-22T03:29:30.969 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:31.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:31.087+0000 7f3b1b157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:31.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:31.092+0000 7f3b1b157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:31.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 8363 8347 1.23029 1.14062 0.0890573 0.10143 2024-09-22T03:29:32.909 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:32.909 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:32.909 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:29:32.909 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:29:32.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 8506 8490 1.22819 1.11719 0.0579534 0.101668 2024-09-22T03:29:33.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.042+0000 7f2d5ba2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.052+0000 7f2d5ba2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.079 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:33.079 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:33.080 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:33.082 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:33.083 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:33.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:29:33.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.192+0000 7f591caa9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.203+0000 7f591caa9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:33.257 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:33.261 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:33.261 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:33.261 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:29:33.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.349+0000 7ffa7006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.357+0000 7ffa7006c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.430+0000 7f15dbe6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.439+0000 7f15dbe6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.497 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:33.498 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:33.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:33.502 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:33.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:29:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.630+0000 7f0eda717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:33.639+0000 7f0eda717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:33.655 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:33.658 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:33.659 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:33.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 8662 8646 1.22802 1.21875 0.111206 0.101691 2024-09-22T03:29:34.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 8822 8806 1.22841 1.25 0.0647345 0.101652 2024-09-22T03:29:35.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:35.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:35.668+0000 7fba4286c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:35.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:35.675+0000 7fba4286c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:35.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 8969 8953 1.227 1.14844 0.0496561 0.101668 2024-09-22T03:29:36.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 9112 9096 1.22511 1.11719 0.0730727 0.101945 2024-09-22T03:29:37.915 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:37.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 9265 9249 1.2246 1.19531 0.199233 0.101987 2024-09-22T03:29:38.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:38.036+0000 7feae4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:38.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:38.041+0000 7feae4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:38.660 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:38.660 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:38.660 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:29:38.660 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:29:38.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:38.812+0000 7f04c2b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:38.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:38.826+0000 7f04c2b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:38.858 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:38.859 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:38.862 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:38.862 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:38.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:29:38.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:29:38.960167+0000 min lat: 0.0127648 max lat: 0.472956 avg lat: 0.102229 2024-09-22T03:29:38.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:29:38.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 9394 9378 1.22099 1.00781 0.0736099 0.102229 2024-09-22T03:29:38.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:38.986+0000 7ff06d46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:38.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:38.997+0000 7ff06d46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:39.088 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:39.089 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:39.089 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:39.091 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:39.091 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:39.091 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:29:39.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:39.200+0000 7f5005a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:39.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:39.209+0000 7f5005a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:39.224 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:39.225 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:39.227 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:39.228 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:39.228 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:29:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:39.343+0000 7f6c2fb4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:39.353+0000 7f6c2fb4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:39.375 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:39.378 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:39.379 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:39.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 9513 9497 1.21621 0.929688 0.223827 0.102667 2024-09-22T03:29:40.203 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:40.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:40.321+0000 7f255ef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:40.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:40.326+0000 7f255ef17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:40.960 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 9660 9644 1.21511 1.14844 0.077459 0.102737 2024-09-22T03:29:41.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 9781 9765 1.21083 0.945312 0.300296 0.102963 2024-09-22T03:29:42.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:42.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:42.693+0000 7fce1390e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:42.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:42.700+0000 7fce1390e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:42.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 15 9937 9922 1.21107 1.22656 0.0747831 0.103107 2024-09-22T03:29:43.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 10095 10079 1.21131 1.22656 0.0327765 0.103095 2024-09-22T03:29:44.379 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:44.379 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:44.380 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:44.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:44.516+0000 7fa6b3a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:44.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:44.525+0000 7fa6b3a58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:44.796 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:29:44.797 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:29:44.811 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-22T03:29:44.008850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80724531,"num_objects":1227,"num_object_clones":0,"num_object_copies":3681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9921,"num_write_kb":80152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9866,"ondisk_log_size":9866,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":490152,"kb_used_data":490152,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208700760,"statfs":{"total":214211493888,"available":213709578240,"internally_reserved":0,"allocated":501915648,"data_stored":501915648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11608064,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1417,"num_write_kb":11336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002204"},"pg_stats":[{"pgid":"6.f","version":"60'639","reported_seq":652,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.771846+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:29:41.771846+0000","last_peered":"2024-09-22T03:29:41.771846+0000","last_clean":"2024-09-22T03:29:41.771846+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:29:41.771846+0000","last_undegraded":"2024-09-22T03:29:41.771846+0000","last_fullsized":"2024-09-22T03:29:41.771846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'551","reported_seq":564,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.917882+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:29:41.917882+0000","last_peered":"2024-09-22T03:29:41.917882+0000","last_clean":"2024-09-22T03:29:41.917882+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:29:41.917882+0000","last_undegraded":"2024-09-22T03:29:41.917882+0000","last_fullsized":"2024-09-22T03:29:41.917882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'583","reported_seq":596,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:40.807012+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:29:40.807012+0000","last_peered":"2024-09-22T03:29:40.807012+0000","last_clean":"2024-09-22T03:29:40.807012+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:29:40.807012+0000","last_undegraded":"2024-09-22T03:29:40.807012+0000","last_fullsized":"2024-09-22T03:29:40.807012+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'735","reported_seq":748,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.590776+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:29:41.590776+0000","last_peered":"2024-09-22T03:29:41.590776+0000","last_clean":"2024-09-22T03:29:41.590776+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:29:41.590776+0000","last_undegraded":"2024-09-22T03:29:41.590776+0000","last_fullsized":"2024-09-22T03:29:41.590776+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'559","reported_seq":572,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:40.400988+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:29:40.400988+0000","last_peered":"2024-09-22T03:29:40.400988+0000","last_clean":"2024-09-22T03:29:40.400988+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:29:40.400988+0000","last_undegraded":"2024-09-22T03:29:40.400988+0000","last_fullsized":"2024-09-22T03:29:40.400988+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'647","reported_seq":660,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.560452+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:29:41.560452+0000","last_peered":"2024-09-22T03:29:41.560452+0000","last_clean":"2024-09-22T03:29:41.560452+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:29:41.560452+0000","last_undegraded":"2024-09-22T03:29:41.560452+0000","last_fullsized":"2024-09-22T03:29:41.560452+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'535","reported_seq":548,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.373318+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:29:41.373318+0000","last_peered":"2024-09-22T03:29:41.373318+0000","last_clean":"2024-09-22T03:29:41.373318+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:29:41.373318+0000","last_undegraded":"2024-09-22T03:29:41.373318+0000","last_fullsized":"2024-09-22T03:29:41.373318+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'557","reported_seq":570,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.957009+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:29:41.957009+0000","last_peered":"2024-09-22T03:29:41.957009+0000","last_clean":"2024-09-22T03:29:41.957009+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:29:41.957009+0000","last_undegraded":"2024-09-22T03:29:41.957009+0000","last_fullsized":"2024-09-22T03:29:41.957009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571136,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":558,"num_write_kb":4464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'567","reported_seq":580,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:38.467814+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:29:38.467814+0000","last_peered":"2024-09-22T03:29:38.467814+0000","last_clean":"2024-09-22T03:29:38.467814+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:29:38.467814+0000","last_undegraded":"2024-09-22T03:29:38.467814+0000","last_fullsized":"2024-09-22T03:29:38.467814+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'655","reported_seq":668,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:40.529110+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:29:40.529110+0000","last_peered":"2024-09-22T03:29:40.529110+0000","last_clean":"2024-09-22T03:29:40.529110+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:29:40.529110+0000","last_undegraded":"2024-09-22T03:29:40.529110+0000","last_fullsized":"2024-09-22T03:29:40.529110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'687","reported_seq":700,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.149400+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:29:41.149400+0000","last_peered":"2024-09-22T03:29:41.149400+0000","last_clean":"2024-09-22T03:29:41.149400+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:29:41.149400+0000","last_undegraded":"2024-09-22T03:29:41.149400+0000","last_fullsized":"2024-09-22T03:29:41.149400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'599","reported_seq":612,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:40.298830+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:29:40.298830+0000","last_peered":"2024-09-22T03:29:40.298830+0000","last_clean":"2024-09-22T03:29:40.298830+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:29:40.298830+0000","last_undegraded":"2024-09-22T03:29:40.298830+0000","last_fullsized":"2024-09-22T03:29:40.298830+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'623","reported_seq":636,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.466493+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:29:41.466493+0000","last_peered":"2024-09-22T03:29:41.466493+0000","last_clean":"2024-09-22T03:29:41.466493+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:29:41.466493+0000","last_undegraded":"2024-09-22T03:29:41.466493+0000","last_fullsized":"2024-09-22T03:29:41.466493+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'559","reported_seq":572,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:40.862353+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:29:40.862353+0000","last_peered":"2024-09-22T03:29:40.862353+0000","last_clean":"2024-09-22T03:29:40.862353+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:29:40.862353+0000","last_undegraded":"2024-09-22T03:29:40.862353+0000","last_fullsized":"2024-09-22T03:29:40.862353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'663","reported_seq":676,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.873314+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:29:41.873314+0000","last_peered":"2024-09-22T03:29:41.873314+0000","last_clean":"2024-09-22T03:29:41.873314+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:29:41.873314+0000","last_undegraded":"2024-09-22T03:29:41.873314+0000","last_fullsized":"2024-09-22T03:29:41.873314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'607","reported_seq":620,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:41.329623+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:29:41.329623+0000","last_peered":"2024-09-22T03:29:41.329623+0000","last_clean":"2024-09-22T03:29:41.329623+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:29:41.329623+0000","last_undegraded":"2024-09-22T03:29:41.329623+0000","last_fullsized":"2024-09-22T03:29:41.329623+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":80134144,"num_objects":1223,"num_object_clones":0,"num_object_copies":3669,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9782,"num_write_kb":78256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213703950336,"internally_reserved":0,"allocated":507543552,"data_stored":507543552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9766,"ondisk_log_size":9766,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213703950336,"internally_reserved":0,"allocated":507543552,"data_stored":507543552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53436407808,"internally_reserved":0,"allocated":116465664,"data_stored":116465664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":112572,"kb_used_data":112572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52185156,"statfs":{"total":53552873472,"available":53437599744,"internally_reserved":0,"allocated":115273728,"data_stored":115273728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":113000,"kb_used_data":113000,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52184728,"statfs":{"total":53552873472,"available":53437161472,"internally_reserved":0,"allocated":115712000,"data_stored":115712000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":131580,"kb_used_data":131580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166148,"statfs":{"total":53552873472,"available":53418135552,"internally_reserved":0,"allocated":134737920,"data_stored":134737920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":133000,"kb_used_data":133000,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52164728,"statfs":{"total":53552873472,"available":53416681472,"internally_reserved":0,"allocated":136192000,"data_stored":136192000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53436407808,"internally_reserved":0,"allocated":116465664,"data_stored":116465664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53415231488,"internally_reserved":0,"allocated":137641984,"data_stored":137641984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53416906752,"internally_reserved":0,"allocated":135966720,"data_stored":135966720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53435404288,"internally_reserved":0,"allocated":117469184,"data_stored":117469184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53436407808,"internally_reserved":0,"allocated":116465664,"data_stored":116465664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53415231488,"internally_reserved":0,"allocated":137641984,"data_stored":137641984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53416906752,"internally_reserved":0,"allocated":135966720,"data_stored":135966720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53435404288,"internally_reserved":0,"allocated":117469184,"data_stored":117469184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53436407808,"internally_reserved":0,"allocated":116465664,"data_stored":116465664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:44.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:44.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 10248 10232 1.21107 1.19531 0.0494223 0.103065 2024-09-22T03:29:45.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:45.056+0000 7f84d8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:45.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:45.062+0000 7f84d8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:45.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 10384 10368 1.20885 1.0625 0.0768739 0.103275 2024-09-22T03:29:46.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 10549 10533 1.21003 1.28906 0.110256 0.103248 2024-09-22T03:29:47.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:47.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:47.397+0000 7f6a61ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:47.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:47.404+0000 7f6a61ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:47.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 10698 10682 1.20936 1.16406 0.185789 0.103216 2024-09-22T03:29:48.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 10835 10819 1.20737 1.07031 0.0904011 0.10344 2024-09-22T03:29:49.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:49.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:49.731+0000 7f6cc4f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:49.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:49.738+0000 7f6cc4f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:49.814 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:49.814 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:49.815 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:29:49.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:49.941+0000 7ffafb757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:49.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:49.953+0000 7ffafb757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:49.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 10986 10970 1.20698 1.17969 0.165618 0.103483 2024-09-22T03:29:50.247 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:29:50.248 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:29:50.261 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-22T03:29:50.010044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87015987,"num_objects":1323,"num_object_clones":0,"num_object_copies":3969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10689,"num_write_kb":86296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10634,"ondisk_log_size":10634,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":525700,"kb_used_data":525700,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208665212,"statfs":{"total":214211493888,"available":213673177088,"internally_reserved":0,"allocated":538316800,"data_stored":538316800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11780096,"num_objects":180,"num_object_clones":0,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1438,"num_write_kb":11504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002441"},"pg_stats":[{"pgid":"6.f","version":"60'685","reported_seq":698,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.970638+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:29:46.970638+0000","last_peered":"2024-09-22T03:29:46.970638+0000","last_clean":"2024-09-22T03:29:46.970638+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:29:46.970638+0000","last_undegraded":"2024-09-22T03:29:46.970638+0000","last_fullsized":"2024-09-22T03:29:46.970638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5619712,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":686,"num_write_kb":5488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'575","reported_seq":588,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.754475+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:29:46.754475+0000","last_peered":"2024-09-22T03:29:46.754475+0000","last_clean":"2024-09-22T03:29:46.754475+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:29:46.754475+0000","last_undegraded":"2024-09-22T03:29:46.754475+0000","last_fullsized":"2024-09-22T03:29:46.754475+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'631","reported_seq":644,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.902948+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:29:46.902948+0000","last_peered":"2024-09-22T03:29:46.902948+0000","last_clean":"2024-09-22T03:29:46.902948+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:29:46.902948+0000","last_undegraded":"2024-09-22T03:29:46.902948+0000","last_fullsized":"2024-09-22T03:29:46.902948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'783","reported_seq":796,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.736707+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:29:46.736707+0000","last_peered":"2024-09-22T03:29:46.736707+0000","last_clean":"2024-09-22T03:29:46.736707+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:29:46.736707+0000","last_undegraded":"2024-09-22T03:29:46.736707+0000","last_fullsized":"2024-09-22T03:29:46.736707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'591","reported_seq":604,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.630640+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:29:46.630640+0000","last_peered":"2024-09-22T03:29:46.630640+0000","last_clean":"2024-09-22T03:29:46.630640+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:29:46.630640+0000","last_undegraded":"2024-09-22T03:29:46.630640+0000","last_fullsized":"2024-09-22T03:29:46.630640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'727","reported_seq":740,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.657262+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:29:46.657262+0000","last_peered":"2024-09-22T03:29:46.657262+0000","last_clean":"2024-09-22T03:29:46.657262+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:29:46.657262+0000","last_undegraded":"2024-09-22T03:29:46.657262+0000","last_fullsized":"2024-09-22T03:29:46.657262+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'583","reported_seq":596,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.580003+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:29:46.580003+0000","last_peered":"2024-09-22T03:29:46.580003+0000","last_clean":"2024-09-22T03:29:46.580003+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:29:46.580003+0000","last_undegraded":"2024-09-22T03:29:46.580003+0000","last_fullsized":"2024-09-22T03:29:46.580003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'615","reported_seq":628,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.848855+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:29:46.848855+0000","last_peered":"2024-09-22T03:29:46.848855+0000","last_clean":"2024-09-22T03:29:46.848855+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:29:46.848855+0000","last_undegraded":"2024-09-22T03:29:46.848855+0000","last_fullsized":"2024-09-22T03:29:46.848855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'599","reported_seq":612,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:45.588625+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:29:45.588625+0000","last_peered":"2024-09-22T03:29:45.588625+0000","last_clean":"2024-09-22T03:29:45.588625+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:29:45.588625+0000","last_undegraded":"2024-09-22T03:29:45.588625+0000","last_fullsized":"2024-09-22T03:29:45.588625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'727","reported_seq":740,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.480636+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:29:46.480636+0000","last_peered":"2024-09-22T03:29:46.480636+0000","last_clean":"2024-09-22T03:29:46.480636+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:29:46.480636+0000","last_undegraded":"2024-09-22T03:29:46.480636+0000","last_fullsized":"2024-09-22T03:29:46.480636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'719","reported_seq":732,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.240881+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:29:46.240881+0000","last_peered":"2024-09-22T03:29:46.240881+0000","last_clean":"2024-09-22T03:29:46.240881+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:29:46.240881+0000","last_undegraded":"2024-09-22T03:29:46.240881+0000","last_fullsized":"2024-09-22T03:29:46.240881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'639","reported_seq":652,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:45.204611+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:29:45.204611+0000","last_peered":"2024-09-22T03:29:45.204611+0000","last_clean":"2024-09-22T03:29:45.204611+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:29:45.204611+0000","last_undegraded":"2024-09-22T03:29:45.204611+0000","last_fullsized":"2024-09-22T03:29:45.204611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'655","reported_seq":668,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:44.786340+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:29:44.786340+0000","last_peered":"2024-09-22T03:29:44.786340+0000","last_clean":"2024-09-22T03:29:44.786340+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:29:44.786340+0000","last_undegraded":"2024-09-22T03:29:44.786340+0000","last_fullsized":"2024-09-22T03:29:44.786340+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'599","reported_seq":612,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.800643+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:29:46.800643+0000","last_peered":"2024-09-22T03:29:46.800643+0000","last_clean":"2024-09-22T03:29:46.800643+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:29:46.800643+0000","last_undegraded":"2024-09-22T03:29:46.800643+0000","last_fullsized":"2024-09-22T03:29:46.800643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'735","reported_seq":748,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.567105+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:29:46.567105+0000","last_peered":"2024-09-22T03:29:46.567105+0000","last_clean":"2024-09-22T03:29:46.567105+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:29:46.567105+0000","last_undegraded":"2024-09-22T03:29:46.567105+0000","last_fullsized":"2024-09-22T03:29:46.567105+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'671","reported_seq":684,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:29:46.955213+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:29:46.955213+0000","last_peered":"2024-09-22T03:29:46.955213+0000","last_clean":"2024-09-22T03:29:46.955213+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:29:46.955213+0000","last_undegraded":"2024-09-22T03:29:46.955213+0000","last_fullsized":"2024-09-22T03:29:46.955213+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":86425600,"num_objects":1319,"num_object_clones":0,"num_object_copies":3957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":84400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213664710656,"internally_reserved":0,"allocated":546783232,"data_stored":546783232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10534,"ondisk_log_size":10534,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213664718848,"internally_reserved":0,"allocated":546775040,"data_stored":546775040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53427748864,"internally_reserved":0,"allocated":125124608,"data_stored":125124608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":120364,"kb_used_data":120364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52177364,"statfs":{"total":53552873472,"available":53429620736,"internally_reserved":0,"allocated":123252736,"data_stored":123252736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":121452,"kb_used_data":121452,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52176276,"statfs":{"total":53552873472,"available":53428506624,"internally_reserved":0,"allocated":124366848,"data_stored":124366848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":140916,"kb_used_data":140916,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52156812,"statfs":{"total":53552873472,"available":53408575488,"internally_reserved":0,"allocated":144297984,"data_stored":144297984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":142968,"kb_used_data":142968,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52154760,"statfs":{"total":53552873472,"available":53406474240,"internally_reserved":0,"allocated":146399232,"data_stored":146399232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53427748864,"internally_reserved":0,"allocated":125124608,"data_stored":125124608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53404262400,"internally_reserved":0,"allocated":148611072,"data_stored":148611072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53405990912,"internally_reserved":0,"allocated":146882560,"data_stored":146882560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53426716672,"internally_reserved":0,"allocated":126156800,"data_stored":126156800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53427748864,"internally_reserved":0,"allocated":125124608,"data_stored":125124608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53404262400,"internally_reserved":0,"allocated":148611072,"data_stored":148611072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53405982720,"internally_reserved":0,"allocated":146890752,"data_stored":146890752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53426716672,"internally_reserved":0,"allocated":126156800,"data_stored":126156800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53427748864,"internally_reserved":0,"allocated":125124608,"data_stored":125124608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:29:50.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 11142 11126 1.20714 1.21875 0.0661003 0.103459 2024-09-22T03:29:51.961 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 11291 11275 1.20655 1.16406 0.0972747 0.103529 2024-09-22T03:29:52.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:52.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:52.145+0000 7f21a7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:52.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:52.151+0000 7f21a7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:52.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 11418 11402 1.20365 0.992188 0.0567452 0.103736 2024-09-22T03:29:53.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 11571 11555 1.20354 1.19531 0.0921137 0.103754 2024-09-22T03:29:54.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:54.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:54.472+0000 7fe551ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:54.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:54.480+0000 7fe551ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:54.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 11704 11688 1.20137 1.03906 0.0423495 0.103896 2024-09-22T03:29:55.263 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:29:55.264 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:29:55.264 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:29:55.264 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:29:55.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.378+0000 7ff9a3e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.389+0000 7ff9a3e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:55.421 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:55.422 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:55.422 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:55.424 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:55.424 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:55.424 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:29:55.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.552+0000 7f890be6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.563+0000 7f890be6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:55.651 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:55.651 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:55.653 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:55.653 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:55.654 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:29:55.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.762+0000 7fee9ceb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.771+0000 7fee9ceb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.813 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:55.814 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:55.817 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:55.817 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:55.818 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:29:55.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.936+0000 7f8090b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:55.947+0000 7f8090b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:55.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 11826 11810 1.19815 0.953125 0.117282 0.104205 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:29:55.963 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:29:55.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:29:55.966 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:29:56.733 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:29:56.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:56.881+0000 7fcb74797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:56.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:56.887+0000 7fcb74797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:56.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 16 11986 11970 1.19881 1.25 0.177216 0.104173 2024-09-22T03:29:57.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 12133 12117 1.19817 1.14844 0.0631237 0.104237 2024-09-22T03:29:58.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:29:58.961942+0000 min lat: 0.0127648 max lat: 0.472956 avg lat: 0.104359 2024-09-22T03:29:58.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:29:58.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 12266 12250 1.19618 1.03906 0.0642064 0.104359 2024-09-22T03:29:59.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:29:59.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:59.162+0000 7f8a49797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:59.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:29:59.168+0000 7f8a49797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:29:59.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 12412 12396 1.1955 1.14062 0.0754073 0.104463 2024-09-22T03:30:00.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 12550 12534 1.19406 1.07812 0.0874424 0.104591 2024-09-22T03:30:00.967 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:00.967 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:30:01.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:01.090+0000 7f0b44397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:01.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:01.100+0000 7f0b44397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:01.380 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:01.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:01.548+0000 7fc62e42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:01.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:01.554+0000 7fc62e42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:01.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 12718 12702 1.19549 1.3125 0.0867708 0.104463 2024-09-22T03:30:02.076 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:30:02.098 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:30:02.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:02.247+0000 7f1be42d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:02.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:02.263+0000 7f1be42d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:02.962 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 12840 12824 1.1926 0.953125 0.232083 0.104618 2024-09-22T03:30:03.077 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:30:03.098 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:30:03.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:03.212+0000 7f0643afa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:03.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:03.223+0000 7f0643afa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:03.766 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:03.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:03.890+0000 7f914e70e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:03.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:03.897+0000 7f914e70e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:03.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 12982 12966 1.19162 1.10938 0.0958465 0.104815 2024-09-22T03:30:04.081 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:30:04.098 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:30:04.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:04.230+0000 7fe10e117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:04.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:04.241+0000 7fe10e117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:04.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 13111 13095 1.18948 1.00781 0.0662222 0.105017 2024-09-22T03:30:05.085 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:30:05.108 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:30:05.108 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:30:05.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:05.241+0000 7f7f2ee2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:05.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:05.251+0000 7f7f2ee2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:30:05.485 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:30:05.501 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-22T03:30:05.085227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":104735283,"num_objects":1594,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12852,"num_write_kb":103600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12797,"ondisk_log_size":12797,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":634560,"kb_used_data":634560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208556352,"statfs":{"total":214211493888,"available":213561704448,"internally_reserved":0,"allocated":649789440,"data_stored":649789440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5996544,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.073405"},"pg_stats":[{"pgid":"6.f","version":"60'783","reported_seq":796,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:00.647104+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:30:00.647104+0000","last_peered":"2024-09-22T03:30:00.647104+0000","last_clean":"2024-09-22T03:30:00.647104+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:30:00.647104+0000","last_undegraded":"2024-09-22T03:30:00.647104+0000","last_fullsized":"2024-09-22T03:30:00.647104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'687","reported_seq":700,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.886296+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:30:01.886296+0000","last_peered":"2024-09-22T03:30:01.886296+0000","last_clean":"2024-09-22T03:30:01.886296+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:30:01.886296+0000","last_undegraded":"2024-09-22T03:30:01.886296+0000","last_fullsized":"2024-09-22T03:30:01.886296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'775","reported_seq":788,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.703443+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:30:01.703443+0000","last_peered":"2024-09-22T03:30:01.703443+0000","last_clean":"2024-09-22T03:30:01.703443+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:30:01.703443+0000","last_undegraded":"2024-09-22T03:30:01.703443+0000","last_fullsized":"2024-09-22T03:30:01.703443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'967","reported_seq":980,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.933182+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:30:01.933182+0000","last_peered":"2024-09-22T03:30:01.933182+0000","last_clean":"2024-09-22T03:30:01.933182+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:30:01.933182+0000","last_undegraded":"2024-09-22T03:30:01.933182+0000","last_fullsized":"2024-09-22T03:30:01.933182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'679","reported_seq":692,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:00.781871+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:30:00.781871+0000","last_peered":"2024-09-22T03:30:00.781871+0000","last_clean":"2024-09-22T03:30:00.781871+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:30:00.781871+0000","last_undegraded":"2024-09-22T03:30:00.781871+0000","last_fullsized":"2024-09-22T03:30:00.781871+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'856","reported_seq":869,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.483548+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:30:01.483548+0000","last_peered":"2024-09-22T03:30:01.483548+0000","last_clean":"2024-09-22T03:30:01.483548+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:30:01.483548+0000","last_undegraded":"2024-09-22T03:30:01.483548+0000","last_fullsized":"2024-09-22T03:30:01.483548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7020544,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":857,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'695","reported_seq":708,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.779210+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:30:01.779210+0000","last_peered":"2024-09-22T03:30:01.779210+0000","last_clean":"2024-09-22T03:30:01.779210+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:30:01.779210+0000","last_undegraded":"2024-09-22T03:30:01.779210+0000","last_fullsized":"2024-09-22T03:30:01.779210+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'775","reported_seq":788,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.121801+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:30:01.121801+0000","last_peered":"2024-09-22T03:30:01.121801+0000","last_clean":"2024-09-22T03:30:01.121801+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:30:01.121801+0000","last_undegraded":"2024-09-22T03:30:01.121801+0000","last_fullsized":"2024-09-22T03:30:01.121801+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'759","reported_seq":772,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.839448+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:30:01.839448+0000","last_peered":"2024-09-22T03:30:01.839448+0000","last_clean":"2024-09-22T03:30:01.839448+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:30:01.839448+0000","last_undegraded":"2024-09-22T03:30:01.839448+0000","last_fullsized":"2024-09-22T03:30:01.839448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'895","reported_seq":908,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.321447+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:30:01.321447+0000","last_peered":"2024-09-22T03:30:01.321447+0000","last_clean":"2024-09-22T03:30:01.321447+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:30:01.321447+0000","last_undegraded":"2024-09-22T03:30:01.321447+0000","last_fullsized":"2024-09-22T03:30:01.321447+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'871","reported_seq":884,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:00.352637+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:30:00.352637+0000","last_peered":"2024-09-22T03:30:00.352637+0000","last_clean":"2024-09-22T03:30:00.352637+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:30:00.352637+0000","last_undegraded":"2024-09-22T03:30:00.352637+0000","last_fullsized":"2024-09-22T03:30:00.352637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'791","reported_seq":804,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.604698+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:30:01.604698+0000","last_peered":"2024-09-22T03:30:01.604698+0000","last_clean":"2024-09-22T03:30:01.604698+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:30:01.604698+0000","last_undegraded":"2024-09-22T03:30:01.604698+0000","last_fullsized":"2024-09-22T03:30:01.604698+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'767","reported_seq":780,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.447863+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:30:01.447863+0000","last_peered":"2024-09-22T03:30:01.447863+0000","last_clean":"2024-09-22T03:30:01.447863+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:30:01.447863+0000","last_undegraded":"2024-09-22T03:30:01.447863+0000","last_fullsized":"2024-09-22T03:30:01.447863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'751","reported_seq":764,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.670077+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:30:01.670077+0000","last_peered":"2024-09-22T03:30:01.670077+0000","last_clean":"2024-09-22T03:30:01.670077+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:30:01.670077+0000","last_undegraded":"2024-09-22T03:30:01.670077+0000","last_fullsized":"2024-09-22T03:30:01.670077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'863","reported_seq":876,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.794478+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:30:01.794478+0000","last_peered":"2024-09-22T03:30:01.794478+0000","last_clean":"2024-09-22T03:30:01.794478+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:30:01.794478+0000","last_undegraded":"2024-09-22T03:30:01.794478+0000","last_fullsized":"2024-09-22T03:30:01.794478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'783","reported_seq":796,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.352061+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:30:01.352061+0000","last_peered":"2024-09-22T03:30:01.352061+0000","last_clean":"2024-09-22T03:30:01.352061+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:30:01.352061+0000","last_undegraded":"2024-09-22T03:30:01.352061+0000","last_fullsized":"2024-09-22T03:30:01.352061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":104144896,"num_objects":1590,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12713,"num_write_kb":101704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213553717248,"internally_reserved":0,"allocated":657776640,"data_stored":657776640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12697,"ondisk_log_size":12697,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213553717248,"internally_reserved":0,"allocated":657776640,"data_stored":657776640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":145880,"kb_used_data":145880,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52151848,"statfs":{"total":53552873472,"available":53403492352,"internally_reserved":0,"allocated":149381120,"data_stored":149381120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":145624,"kb_used_data":145624,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52152104,"statfs":{"total":53552873472,"available":53403754496,"internally_reserved":0,"allocated":149118976,"data_stored":149118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":170332,"kb_used_data":170332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52127396,"statfs":{"total":53552873472,"available":53378453504,"internally_reserved":0,"allocated":174419968,"data_stored":174419968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":172724,"kb_used_data":172724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125004,"statfs":{"total":53552873472,"available":53376004096,"internally_reserved":0,"allocated":176869376,"data_stored":176869376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53373952000,"internally_reserved":0,"allocated":178921472,"data_stored":178921472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53376057344,"internally_reserved":0,"allocated":176816128,"data_stored":176816128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53402128384,"internally_reserved":0,"allocated":150745088,"data_stored":150745088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53373952000,"internally_reserved":0,"allocated":178921472,"data_stored":178921472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53376057344,"internally_reserved":0,"allocated":176816128,"data_stored":176816128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53402128384,"internally_reserved":0,"allocated":150745088,"data_stored":150745088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:30:05.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:30:05.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:05.653+0000 7fbbbb517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:05.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:05.661+0000 7fbbbb517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:05.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 13242 13226 1.18757 1.02344 0.0840124 0.105162 2024-09-22T03:30:05.983 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:30:05.984 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:30:06.001 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-22T03:30:05.085227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":104735283,"num_objects":1594,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12852,"num_write_kb":103600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12797,"ondisk_log_size":12797,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":634560,"kb_used_data":634560,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208556352,"statfs":{"total":214211493888,"available":213561704448,"internally_reserved":0,"allocated":649789440,"data_stored":649789440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5996544,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.073405"},"pg_stats":[{"pgid":"6.f","version":"60'783","reported_seq":796,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:00.647104+0000","last_change":"2024-09-22T03:28:38.045771+0000","last_active":"2024-09-22T03:30:00.647104+0000","last_peered":"2024-09-22T03:30:00.647104+0000","last_clean":"2024-09-22T03:30:00.647104+0000","last_became_active":"2024-09-22T03:28:38.037525+0000","last_became_peered":"2024-09-22T03:28:38.037525+0000","last_unstale":"2024-09-22T03:30:00.647104+0000","last_undegraded":"2024-09-22T03:30:00.647104+0000","last_fullsized":"2024-09-22T03:30:00.647104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"60'687","reported_seq":700,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.886296+0000","last_change":"2024-09-22T03:28:38.022155+0000","last_active":"2024-09-22T03:30:01.886296+0000","last_peered":"2024-09-22T03:30:01.886296+0000","last_clean":"2024-09-22T03:30:01.886296+0000","last_became_active":"2024-09-22T03:28:37.995196+0000","last_became_peered":"2024-09-22T03:28:37.995196+0000","last_unstale":"2024-09-22T03:30:01.886296+0000","last_undegraded":"2024-09-22T03:30:01.886296+0000","last_fullsized":"2024-09-22T03:30:01.886296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"60'775","reported_seq":788,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.703443+0000","last_change":"2024-09-22T03:28:38.008117+0000","last_active":"2024-09-22T03:30:01.703443+0000","last_peered":"2024-09-22T03:30:01.703443+0000","last_clean":"2024-09-22T03:30:01.703443+0000","last_became_active":"2024-09-22T03:28:37.997808+0000","last_became_peered":"2024-09-22T03:28:37.997808+0000","last_unstale":"2024-09-22T03:30:01.703443+0000","last_undegraded":"2024-09-22T03:30:01.703443+0000","last_fullsized":"2024-09-22T03:30:01.703443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"60'967","reported_seq":980,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.933182+0000","last_change":"2024-09-22T03:28:37.977133+0000","last_active":"2024-09-22T03:30:01.933182+0000","last_peered":"2024-09-22T03:30:01.933182+0000","last_clean":"2024-09-22T03:30:01.933182+0000","last_became_active":"2024-09-22T03:28:37.970065+0000","last_became_peered":"2024-09-22T03:28:37.970065+0000","last_unstale":"2024-09-22T03:30:01.933182+0000","last_undegraded":"2024-09-22T03:30:01.933182+0000","last_fullsized":"2024-09-22T03:30:01.933182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'679","reported_seq":692,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:00.781871+0000","last_change":"2024-09-22T03:28:38.031670+0000","last_active":"2024-09-22T03:30:00.781871+0000","last_peered":"2024-09-22T03:30:00.781871+0000","last_clean":"2024-09-22T03:30:00.781871+0000","last_became_active":"2024-09-22T03:28:38.021701+0000","last_became_peered":"2024-09-22T03:28:38.021701+0000","last_unstale":"2024-09-22T03:30:00.781871+0000","last_undegraded":"2024-09-22T03:30:00.781871+0000","last_fullsized":"2024-09-22T03:30:00.781871+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"60'856","reported_seq":869,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.483548+0000","last_change":"2024-09-22T03:28:37.995360+0000","last_active":"2024-09-22T03:30:01.483548+0000","last_peered":"2024-09-22T03:30:01.483548+0000","last_clean":"2024-09-22T03:30:01.483548+0000","last_became_active":"2024-09-22T03:28:37.989564+0000","last_became_peered":"2024-09-22T03:28:37.989564+0000","last_unstale":"2024-09-22T03:30:01.483548+0000","last_undegraded":"2024-09-22T03:30:01.483548+0000","last_fullsized":"2024-09-22T03:30:01.483548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7020544,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":857,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"60'695","reported_seq":708,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.779210+0000","last_change":"2024-09-22T03:28:38.035299+0000","last_active":"2024-09-22T03:30:01.779210+0000","last_peered":"2024-09-22T03:30:01.779210+0000","last_clean":"2024-09-22T03:30:01.779210+0000","last_became_active":"2024-09-22T03:28:38.030054+0000","last_became_peered":"2024-09-22T03:28:38.030054+0000","last_unstale":"2024-09-22T03:30:01.779210+0000","last_undegraded":"2024-09-22T03:30:01.779210+0000","last_fullsized":"2024-09-22T03:30:01.779210+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"60'775","reported_seq":788,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.121801+0000","last_change":"2024-09-22T03:28:38.051853+0000","last_active":"2024-09-22T03:30:01.121801+0000","last_peered":"2024-09-22T03:30:01.121801+0000","last_clean":"2024-09-22T03:30:01.121801+0000","last_became_active":"2024-09-22T03:28:38.020562+0000","last_became_peered":"2024-09-22T03:28:38.020562+0000","last_unstale":"2024-09-22T03:30:01.121801+0000","last_undegraded":"2024-09-22T03:30:01.121801+0000","last_fullsized":"2024-09-22T03:30:01.121801+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'759","reported_seq":772,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.839448+0000","last_change":"2024-09-22T03:28:37.994081+0000","last_active":"2024-09-22T03:30:01.839448+0000","last_peered":"2024-09-22T03:30:01.839448+0000","last_clean":"2024-09-22T03:30:01.839448+0000","last_became_active":"2024-09-22T03:28:37.989856+0000","last_became_peered":"2024-09-22T03:28:37.989856+0000","last_unstale":"2024-09-22T03:30:01.839448+0000","last_undegraded":"2024-09-22T03:30:01.839448+0000","last_fullsized":"2024-09-22T03:30:01.839448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":95,"num_object_clones":0,"num_object_copies":285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889611+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:28:38.889611+0000","last_peered":"2024-09-22T03:28:38.889611+0000","last_clean":"2024-09-22T03:28:38.889611+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:28:38.889611+0000","last_undegraded":"2024-09-22T03:28:38.889611+0000","last_fullsized":"2024-09-22T03:28:38.889611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":149,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889363+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:28:38.889363+0000","last_peered":"2024-09-22T03:28:38.889363+0000","last_clean":"2024-09-22T03:28:38.889363+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:28:38.889363+0000","last_undegraded":"2024-09-22T03:28:38.889363+0000","last_fullsized":"2024-09-22T03:28:38.889363+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"60'895","reported_seq":908,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.321447+0000","last_change":"2024-09-22T03:28:38.101731+0000","last_active":"2024-09-22T03:30:01.321447+0000","last_peered":"2024-09-22T03:30:01.321447+0000","last_clean":"2024-09-22T03:30:01.321447+0000","last_became_active":"2024-09-22T03:28:38.098078+0000","last_became_peered":"2024-09-22T03:28:38.098078+0000","last_unstale":"2024-09-22T03:30:01.321447+0000","last_undegraded":"2024-09-22T03:30:01.321447+0000","last_fullsized":"2024-09-22T03:30:01.321447+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.889112+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:28:38.889112+0000","last_peered":"2024-09-22T03:28:38.889112+0000","last_clean":"2024-09-22T03:28:38.889112+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:28:38.889112+0000","last_undegraded":"2024-09-22T03:28:38.889112+0000","last_fullsized":"2024-09-22T03:28:38.889112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"60'871","reported_seq":884,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:00.352637+0000","last_change":"2024-09-22T03:28:37.995808+0000","last_active":"2024-09-22T03:30:00.352637+0000","last_peered":"2024-09-22T03:30:00.352637+0000","last_clean":"2024-09-22T03:30:00.352637+0000","last_became_active":"2024-09-22T03:28:37.971452+0000","last_became_peered":"2024-09-22T03:28:37.971452+0000","last_unstale":"2024-09-22T03:30:00.352637+0000","last_undegraded":"2024-09-22T03:30:00.352637+0000","last_fullsized":"2024-09-22T03:30:00.352637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.886135+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.886135+0000","last_peered":"2024-09-22T03:28:38.886135+0000","last_clean":"2024-09-22T03:28:38.886135+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:28:38.886135+0000","last_undegraded":"2024-09-22T03:28:38.886135+0000","last_fullsized":"2024-09-22T03:28:38.886135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"60'791","reported_seq":804,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.604698+0000","last_change":"2024-09-22T03:28:37.988900+0000","last_active":"2024-09-22T03:30:01.604698+0000","last_peered":"2024-09-22T03:30:01.604698+0000","last_clean":"2024-09-22T03:30:01.604698+0000","last_became_active":"2024-09-22T03:28:37.973163+0000","last_became_peered":"2024-09-22T03:28:37.973163+0000","last_unstale":"2024-09-22T03:30:01.604698+0000","last_undegraded":"2024-09-22T03:30:01.604698+0000","last_fullsized":"2024-09-22T03:30:01.604698+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":150,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.884669+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:28:38.884669+0000","last_peered":"2024-09-22T03:28:38.884669+0000","last_clean":"2024-09-22T03:28:38.884669+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:28:38.884669+0000","last_undegraded":"2024-09-22T03:28:38.884669+0000","last_fullsized":"2024-09-22T03:28:38.884669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"60'767","reported_seq":780,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.447863+0000","last_change":"2024-09-22T03:28:38.031991+0000","last_active":"2024-09-22T03:30:01.447863+0000","last_peered":"2024-09-22T03:30:01.447863+0000","last_clean":"2024-09-22T03:30:01.447863+0000","last_became_active":"2024-09-22T03:28:38.018473+0000","last_became_peered":"2024-09-22T03:28:38.018473+0000","last_unstale":"2024-09-22T03:30:01.447863+0000","last_undegraded":"2024-09-22T03:30:01.447863+0000","last_fullsized":"2024-09-22T03:30:01.447863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906687+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.906687+0000","last_peered":"2024-09-22T03:28:38.906687+0000","last_clean":"2024-09-22T03:28:38.906687+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:28:38.906687+0000","last_undegraded":"2024-09-22T03:28:38.906687+0000","last_fullsized":"2024-09-22T03:28:38.906687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"60'751","reported_seq":764,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.670077+0000","last_change":"2024-09-22T03:28:38.076478+0000","last_active":"2024-09-22T03:30:01.670077+0000","last_peered":"2024-09-22T03:30:01.670077+0000","last_clean":"2024-09-22T03:30:01.670077+0000","last_became_active":"2024-09-22T03:28:38.072371+0000","last_became_peered":"2024-09-22T03:28:38.072371+0000","last_unstale":"2024-09-22T03:30:01.670077+0000","last_undegraded":"2024-09-22T03:30:01.670077+0000","last_fullsized":"2024-09-22T03:30:01.670077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.892112+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:28:38.892112+0000","last_peered":"2024-09-22T03:28:38.892112+0000","last_clean":"2024-09-22T03:28:38.892112+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:28:38.892112+0000","last_undegraded":"2024-09-22T03:28:38.892112+0000","last_fullsized":"2024-09-22T03:28:38.892112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"60'863","reported_seq":876,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.794478+0000","last_change":"2024-09-22T03:28:38.040857+0000","last_active":"2024-09-22T03:30:01.794478+0000","last_peered":"2024-09-22T03:30:01.794478+0000","last_clean":"2024-09-22T03:30:01.794478+0000","last_became_active":"2024-09-22T03:28:38.032000+0000","last_became_peered":"2024-09-22T03:28:38.032000+0000","last_unstale":"2024-09-22T03:30:01.794478+0000","last_undegraded":"2024-09-22T03:30:01.794478+0000","last_fullsized":"2024-09-22T03:30:01.794478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.906831+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:28:38.906831+0000","last_peered":"2024-09-22T03:28:38.906831+0000","last_clean":"2024-09-22T03:28:38.906831+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:28:38.906831+0000","last_undegraded":"2024-09-22T03:28:38.906831+0000","last_fullsized":"2024-09-22T03:28:38.906831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"60'783","reported_seq":796,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:30:01.352061+0000","last_change":"2024-09-22T03:28:38.041414+0000","last_active":"2024-09-22T03:30:01.352061+0000","last_peered":"2024-09-22T03:30:01.352061+0000","last_clean":"2024-09-22T03:30:01.352061+0000","last_became_active":"2024-09-22T03:28:38.031417+0000","last_became_peered":"2024-09-22T03:28:38.031417+0000","last_unstale":"2024-09-22T03:30:01.352061+0000","last_undegraded":"2024-09-22T03:30:01.352061+0000","last_fullsized":"2024-09-22T03:30:01.352061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:28:36.857520+0000","last_clean_scrub_stamp":"2024-09-22T03:28:36.857520+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-22T03:28:38.888903+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:28:38.888903+0000","last_peered":"2024-09-22T03:28:38.888903+0000","last_clean":"2024-09-22T03:28:38.888903+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:28:38.888903+0000","last_undegraded":"2024-09-22T03:28:38.888903+0000","last_fullsized":"2024-09-22T03:28:38.888903+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":104144896,"num_objects":1590,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12713,"num_write_kb":101704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213553717248,"internally_reserved":0,"allocated":657776640,"data_stored":657776640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12697,"ondisk_log_size":12697,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213553717248,"internally_reserved":0,"allocated":657776640,"data_stored":657776640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":145880,"kb_used_data":145880,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52151848,"statfs":{"total":53552873472,"available":53403492352,"internally_reserved":0,"allocated":149381120,"data_stored":149381120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":145624,"kb_used_data":145624,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52152104,"statfs":{"total":53552873472,"available":53403754496,"internally_reserved":0,"allocated":149118976,"data_stored":149118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":170332,"kb_used_data":170332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52127396,"statfs":{"total":53552873472,"available":53378453504,"internally_reserved":0,"allocated":174419968,"data_stored":174419968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":172724,"kb_used_data":172724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125004,"statfs":{"total":53552873472,"available":53376004096,"internally_reserved":0,"allocated":176869376,"data_stored":176869376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53373952000,"internally_reserved":0,"allocated":178921472,"data_stored":178921472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53376057344,"internally_reserved":0,"allocated":176816128,"data_stored":176816128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53402128384,"internally_reserved":0,"allocated":150745088,"data_stored":150745088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":53552873472,"available":53373952000,"internally_reserved":0,"allocated":178921472,"data_stored":178921472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":53552873472,"available":53376057344,"internally_reserved":0,"allocated":176816128,"data_stored":176816128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":53552873472,"available":53402128384,"internally_reserved":0,"allocated":150745088,"data_stored":150745088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":53552873472,"available":53401579520,"internally_reserved":0,"allocated":151293952,"data_stored":151293952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:30:06.003 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:30:06.003 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:06.004 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:06.004 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:06.103 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:06.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.164+0000 7fb971b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.177+0000 7fb971b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:06.206 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:06.206 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:06.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:06.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.238+0000 7f77bbb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.245+0000 7f77bbb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.324+0000 7f0c69c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.337+0000 7f0c69c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:06.425 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:06.425 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:06.425 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:06.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.575+0000 7fa64e24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.585+0000 7fa64e24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.684 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:06.685 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:06.686 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:06.686 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:06.686 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:06.689 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:06.689 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:06.689 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:06.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.833+0000 7f5e43d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:06.845+0000 7f5e43d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:06.876 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:06.877 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:06.877 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:06.880 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:06.881 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:06.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 13390 13374 1.18722 1.15625 0.0755617 0.105238 2024-09-22T03:30:07.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 13534 13518 1.18652 1.125 0.057512 0.105247 2024-09-22T03:30:08.412 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:08.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:08.543+0000 7f88c1f4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:08.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:08.549+0000 7f88c1f4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:08.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 15 13672 13657 1.1854 1.08594 0.115488 0.105352 2024-09-22T03:30:09.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.0693 2024-09-22T03:30:09.963 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 13672 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.18589 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.113393 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.4375 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 0.929688 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 151 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 14.5143 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 184 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 119 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.105361 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.0594827 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.472956 2024-09-22T03:30:09.964 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0127648 2024-09-22T03:30:09.989 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:30:10.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:10.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:10.849+0000 7fde0106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:10.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:10.857+0000 7fde0106c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:11.881 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:11.881 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:11.881 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:11.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:12.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.027+0000 7f9aaa2ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.043+0000 7f9aaa2ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:12.067 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:12.071 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:12.071 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:12.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:12.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.213+0000 7f9057b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.222+0000 7f9057b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.305 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:12.306 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:12.309 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:12.309 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:12.309 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:12.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.441+0000 7f7247b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.451+0000 7f7247b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.523 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:12.523 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:12.523 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:12.524 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:12.526 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:12.527 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:12.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:12.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.671+0000 7faca1997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:12.685+0000 7faca1997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:12.717 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:12.717 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:12.717 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:12.717 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:12.717 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:12.718 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:12.718 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:12.718 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:12.718 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:12.718 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:12.718 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:12.718 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:12.721 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:12.721 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:13.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:13.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:13.305+0000 7fd0cbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:13.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:13.312+0000 7fd0cbd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:15.570 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:15.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:15.714+0000 7f523fd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:15.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:15.721+0000 7f523fd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:17.722 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:17.723 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:17.723 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:17.723 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:17.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:17.888+0000 7f50b5c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:17.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:17.902+0000 7f50b5c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:17.925 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:17.943 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:17.943 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:17.944 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:17.945 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:17.945 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:17.948 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:17.949 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:17.949 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:18.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.089+0000 7fc6baa2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.096+0000 7fc6baa2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.155+0000 7f307e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.165+0000 7f307e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.251 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:18.251 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:18.251 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:18.251 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:18.251 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:18.251 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:18.252 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:18.252 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:18.252 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:18.252 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:18.252 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:18.252 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:18.256 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:18.256 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:18.257 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.426+0000 7f9f61f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.441+0000 7f9f61f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.494 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:18.495 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:18.499 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:18.500 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:18.500 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:18.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.698+0000 7fcb43b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:18.712+0000 7fcb43b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:18.741 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:18.741 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:18.741 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:18.741 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:18.742 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:18.746 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:18.746 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:20.410 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:20.475 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 1709 objects 2024-09-22T03:30:20.475 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :10.4854 2024-09-22T03:30:20.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:20.524+0000 7ff09f423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:20.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:20.529+0000 7ff09f423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:22.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:22.621 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-22T03:30:22.622 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-22T03:30:22.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:22.714+0000 7fb1cd0fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:22.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:22.721+0000 7fb1cd0fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:22.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:22.760+0000 7f6530197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:22.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:22.771+0000 7f6530197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:23.112 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_3' removed 2024-09-22T03:30:23.134 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:30:23.134 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:30:23.134 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-22T03:30:23.135 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_4 16 2024-09-22T03:30:23.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:23.272+0000 7ff366157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:23.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:23.285+0000 7ff366157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:23.747 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:23.747 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:23.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:30:23.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:23.846+0000 7f4308ace640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:23.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:23.853+0000 7f4308ace640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:30:24.027 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:30:24.040 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-22T03:30:24.016629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":31540,"kb_used_data":31540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209159372,"statfs":{"total":214211493888,"available":214179196928,"internally_reserved":0,"allocated":32296960,"data_stored":32296960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-88670232,"num_objects":-1354,"num_object_clones":0,"num_object_copies":-4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-14029,"num_write_kb":-109377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002035"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":295,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.110273+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:05.110273+0000","last_peered":"2024-09-22T03:30:05.110273+0000","last_clean":"2024-09-22T03:30:05.110273+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:05.110273+0000","last_undegraded":"2024-09-22T03:30:05.110273+0000","last_fullsized":"2024-09-22T03:30:05.110273+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":161,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.112892+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:05.112892+0000","last_peered":"2024-09-22T03:30:05.112892+0000","last_clean":"2024-09-22T03:30:05.112892+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:05.112892+0000","last_undegraded":"2024-09-22T03:30:05.112892+0000","last_fullsized":"2024-09-22T03:30:05.112892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":160,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.109281+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:05.109281+0000","last_peered":"2024-09-22T03:30:05.109281+0000","last_clean":"2024-09-22T03:30:05.109281+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:05.109281+0000","last_undegraded":"2024-09-22T03:30:05.109281+0000","last_fullsized":"2024-09-22T03:30:05.109281+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.109999+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:05.109999+0000","last_peered":"2024-09-22T03:30:05.109999+0000","last_clean":"2024-09-22T03:30:05.109999+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:05.109999+0000","last_undegraded":"2024-09-22T03:30:05.109999+0000","last_fullsized":"2024-09-22T03:30:05.109999+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":162,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.106229+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:05.106229+0000","last_peered":"2024-09-22T03:30:05.106229+0000","last_clean":"2024-09-22T03:30:05.106229+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:05.106229+0000","last_undegraded":"2024-09-22T03:30:05.106229+0000","last_fullsized":"2024-09-22T03:30:05.106229+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":160,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.099130+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:05.099130+0000","last_peered":"2024-09-22T03:30:05.099130+0000","last_clean":"2024-09-22T03:30:05.099130+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:05.099130+0000","last_undegraded":"2024-09-22T03:30:05.099130+0000","last_fullsized":"2024-09-22T03:30:05.099130+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.113051+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:05.113051+0000","last_peered":"2024-09-22T03:30:05.113051+0000","last_clean":"2024-09-22T03:30:05.113051+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:05.113051+0000","last_undegraded":"2024-09-22T03:30:05.113051+0000","last_fullsized":"2024-09-22T03:30:05.113051+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":160,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.099984+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:05.099984+0000","last_peered":"2024-09-22T03:30:05.099984+0000","last_clean":"2024-09-22T03:30:05.099984+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:05.099984+0000","last_undegraded":"2024-09-22T03:30:05.099984+0000","last_fullsized":"2024-09-22T03:30:05.099984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-22T03:30:05.114460+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:05.114460+0000","last_peered":"2024-09-22T03:30:05.114460+0000","last_clean":"2024-09-22T03:30:05.114460+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:05.114460+0000","last_undegraded":"2024-09-22T03:30:05.114460+0000","last_fullsized":"2024-09-22T03:30:05.114460+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214179196928,"internally_reserved":0,"allocated":32296960,"data_stored":32296960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53545181184,"internally_reserved":0,"allocated":7692288,"data_stored":7692288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706103,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7512,"kb_used_data":7512,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290216,"statfs":{"total":53552873472,"available":53545181184,"internally_reserved":0,"allocated":7692288,"data_stored":7692288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706103,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7604,"kb_used_data":7604,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290124,"statfs":{"total":53552873472,"available":53545086976,"internally_reserved":0,"allocated":7786496,"data_stored":7786496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706103,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7640,"kb_used_data":7640,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290088,"statfs":{"total":53552873472,"available":53545050112,"internally_reserved":0,"allocated":7823360,"data_stored":7823360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706103,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8784,"kb_used_data":8784,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52288944,"statfs":{"total":53552873472,"available":53543878656,"internally_reserved":0,"allocated":8994816,"data_stored":8994816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53545181184,"internally_reserved":0,"allocated":7692288,"data_stored":7692288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53543878656,"internally_reserved":0,"allocated":8994816,"data_stored":8994816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53545050112,"internally_reserved":0,"allocated":7823360,"data_stored":7823360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53545086976,"internally_reserved":0,"allocated":7786496,"data_stored":7786496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53545181184,"internally_reserved":0,"allocated":7692288,"data_stored":7692288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:30:24.115 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_4' created 2024-09-22T03:30:24.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-22T03:30:24.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:24.246+0000 7fc60f8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:24.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:24.258+0000 7fc60f8b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:24.811 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:24.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:24.913+0000 7f6c63d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:24.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:24.917+0000 7f6c63d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:25.117 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-22T03:30:26.133 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-22T03:30:26.136 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:30:26.136 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:30:26.191 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:30:26.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:30:26.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_49135 2024-09-22T03:30:26.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:30:26.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:30:27.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:27.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:27.148+0000 7f81ea863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:27.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:27.153+0000 7f81ea863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:27.191 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 170 154 1.20304 1.20312 0.0413746 0.0925395 2024-09-22T03:30:28.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 325 309 1.20691 1.21094 0.028591 0.09569 2024-09-22T03:30:29.042 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:29.042 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:29.042 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:29.043 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:29.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.170+0000 7f6ec1597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.179+0000 7f6ec1597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 470 454 1.18218 1.13281 0.115565 0.103535 2024-09-22T03:30:29.229 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:29.229 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:29.229 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:29.229 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:29.229 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:29.229 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:29.229 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:29.230 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:29.230 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:29.230 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:29.230 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:29.230 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:29.233 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:29.233 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:29.233 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:29.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:29.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.355+0000 7f23db597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.367+0000 7f23db597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.390 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:29.390 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:29.391 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:29.394 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:29.395 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:29.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:29.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.444+0000 7f43fc197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.451+0000 7f43fc197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.552+0000 7efc5486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.565+0000 7efc5486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:29.621 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:29.622 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:29.622 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:29.622 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:29.622 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:29.624 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:29.625 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:29.625 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:29.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.781+0000 7fa864cc3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:29.790+0000 7fa864cc3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:29.842 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:29.843 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:30.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 604 588 1.14833 1.04688 0.0175409 0.107022 2024-09-22T03:30:31.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 742 726 1.13427 1.07812 0.0776229 0.109386 2024-09-22T03:30:31.675 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:31.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:31.795+0000 7f60514d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:31.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:31.800+0000 7f60514d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:32.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 903 887 1.15484 1.25781 0.0687044 0.107206 2024-09-22T03:30:33.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 1065 1049 1.17064 1.26562 0.138553 0.105691 2024-09-22T03:30:33.966 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:34.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:34.082+0000 7fe1f1557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:34.087+0000 7fe1f1557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 16 1219 1203 1.17469 1.20312 0.0635124 0.105426 2024-09-22T03:30:34.844 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:34.844 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:34.844 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:34.844 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:34.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:34.957+0000 7f8756397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:34.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:34.965+0000 7f8756397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:35.016 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:35.017 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:35.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:35.020 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:35.021 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:35.138+0000 7fcd0f717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:35.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:35.147+0000 7fcd0f717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:35.167 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:35.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:35.170 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:35.170 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:35.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1378 1362 1.18218 1.24219 0.106954 0.105034 2024-09-22T03:30:35.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:35.291+0000 7f01b6d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:35.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:35.303+0000 7f01b6d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:35.326 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:35.327 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:35.327 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:35.327 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:35.327 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:35.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:35.329 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:35.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:35.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:35.479+0000 7f6b3f46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:35.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:35.488+0000 7f6b3f46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:35.533 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:35.534 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:35.537 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:35.537 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:36.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 1508 1492 1.16552 1.01562 0.0716586 0.105478 2024-09-22T03:30:36.297 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:36.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:36.408+0000 7f6895757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:36.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:36.412+0000 7f6895757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:37.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 1656 1640 1.16466 1.15625 0.162581 0.106509 2024-09-22T03:30:38.192 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 1806 1790 1.16526 1.17188 0.038954 0.106669 2024-09-22T03:30:38.552 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:38.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:38.679+0000 7fe865c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:38.684+0000 7fe865c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:39.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 1973 1957 1.17597 1.30469 0.104234 0.105672 2024-09-22T03:30:40.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 2111 2095 1.16897 1.07812 0.133745 0.106445 2024-09-22T03:30:40.538 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:40.538 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:30:40.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:40.644+0000 7f3b76157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:40.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:40.654+0000 7f3b76157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:40.863 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:40.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:40.971+0000 7fe7652b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:40.976+0000 7fe7652b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:41.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 2239 2223 1.1577 1 0.126721 0.107274 2024-09-22T03:30:41.752 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:30:41.772 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:30:41.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:41.919+0000 7f2ccc4b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:41.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:41.929+0000 7f2ccc4b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:42.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 2363 2347 1.14589 0.96875 0.06255 0.108463 2024-09-22T03:30:42.753 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:30:42.776 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:30:42.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:42.896+0000 7f218b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:42.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:42.906+0000 7f218b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:43.165 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:43.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 2512 2496 1.14695 1.16406 0.137281 0.108344 2024-09-22T03:30:43.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:43.287+0000 7fbb540d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:43.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:43.294+0000 7fbb540d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:43.758 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:30:43.784 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:30:43.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:43.923+0000 7fae63d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:43.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:43.933+0000 7fae63d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:44.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 2644 2628 1.14052 1.03125 0.148091 0.109144 2024-09-22T03:30:44.769 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:30:44.785 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:30:44.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:30:44.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:44.895+0000 7fa25824f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:44.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:44.903+0000 7fa25824f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.093 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:30:45.094 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:30:45.106 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-22T03:30:44.769789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19743283,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2477,"num_write_kb":20600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2435,"ondisk_log_size":2435,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":119000,"kb_used_data":119000,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209071912,"statfs":{"total":214211493888,"available":214089637888,"internally_reserved":0,"allocated":121856000,"data_stored":121856000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5791744,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":707,"num_write_kb":5656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040001"},"pg_stats":[{"pgid":"7.f","version":"71'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.816152+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:30:41.816152+0000","last_peered":"2024-09-22T03:30:41.816152+0000","last_clean":"2024-09-22T03:30:41.816152+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:30:41.816152+0000","last_undegraded":"2024-09-22T03:30:41.816152+0000","last_fullsized":"2024-09-22T03:30:41.816152+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"71'104","reported_seq":128,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.825573+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:30:41.825573+0000","last_peered":"2024-09-22T03:30:41.825573+0000","last_clean":"2024-09-22T03:30:41.825573+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:30:41.825573+0000","last_undegraded":"2024-09-22T03:30:41.825573+0000","last_fullsized":"2024-09-22T03:30:41.825573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"70'176","reported_seq":200,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.815013+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:30:41.815013+0000","last_peered":"2024-09-22T03:30:41.815013+0000","last_clean":"2024-09-22T03:30:41.815013+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:30:41.815013+0000","last_undegraded":"2024-09-22T03:30:41.815013+0000","last_fullsized":"2024-09-22T03:30:41.815013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"71'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819121+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:30:41.819121+0000","last_peered":"2024-09-22T03:30:41.819121+0000","last_clean":"2024-09-22T03:30:41.819121+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:30:41.819121+0000","last_undegraded":"2024-09-22T03:30:41.819121+0000","last_fullsized":"2024-09-22T03:30:41.819121+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"70'176","reported_seq":200,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.821854+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:30:41.821854+0000","last_peered":"2024-09-22T03:30:41.821854+0000","last_clean":"2024-09-22T03:30:41.821854+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:30:41.821854+0000","last_undegraded":"2024-09-22T03:30:41.821854+0000","last_fullsized":"2024-09-22T03:30:41.821854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"72'159","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.955212+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:30:41.955212+0000","last_peered":"2024-09-22T03:30:41.955212+0000","last_clean":"2024-09-22T03:30:41.955212+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:30:41.955212+0000","last_undegraded":"2024-09-22T03:30:41.955212+0000","last_fullsized":"2024-09-22T03:30:41.955212+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"70'192","reported_seq":216,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.816221+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:30:41.816221+0000","last_peered":"2024-09-22T03:30:41.816221+0000","last_clean":"2024-09-22T03:30:41.816221+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:30:41.816221+0000","last_undegraded":"2024-09-22T03:30:41.816221+0000","last_fullsized":"2024-09-22T03:30:41.816221+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"70'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.828404+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:30:41.828404+0000","last_peered":"2024-09-22T03:30:41.828404+0000","last_clean":"2024-09-22T03:30:41.828404+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:30:41.828404+0000","last_undegraded":"2024-09-22T03:30:41.828404+0000","last_fullsized":"2024-09-22T03:30:41.828404+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"70'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.820342+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:30:41.820342+0000","last_peered":"2024-09-22T03:30:41.820342+0000","last_clean":"2024-09-22T03:30:41.820342+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:30:41.820342+0000","last_undegraded":"2024-09-22T03:30:41.820342+0000","last_fullsized":"2024-09-22T03:30:41.820342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":319,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.817530+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:41.817530+0000","last_peered":"2024-09-22T03:30:41.817530+0000","last_clean":"2024-09-22T03:30:41.817530+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:41.817530+0000","last_undegraded":"2024-09-22T03:30:41.817530+0000","last_fullsized":"2024-09-22T03:30:41.817530+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":185,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.829327+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:41.829327+0000","last_peered":"2024-09-22T03:30:41.829327+0000","last_clean":"2024-09-22T03:30:41.829327+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:41.829327+0000","last_undegraded":"2024-09-22T03:30:41.829327+0000","last_fullsized":"2024-09-22T03:30:41.829327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"71'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.827259+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:30:41.827259+0000","last_peered":"2024-09-22T03:30:41.827259+0000","last_clean":"2024-09-22T03:30:41.827259+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:30:41.827259+0000","last_undegraded":"2024-09-22T03:30:41.827259+0000","last_fullsized":"2024-09-22T03:30:41.827259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.774204+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:41.774204+0000","last_peered":"2024-09-22T03:30:41.774204+0000","last_clean":"2024-09-22T03:30:41.774204+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:41.774204+0000","last_undegraded":"2024-09-22T03:30:41.774204+0000","last_fullsized":"2024-09-22T03:30:41.774204+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"72'127","reported_seq":152,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.914682+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:30:41.914682+0000","last_peered":"2024-09-22T03:30:41.914682+0000","last_clean":"2024-09-22T03:30:41.914682+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:30:41.914682+0000","last_undegraded":"2024-09-22T03:30:41.914682+0000","last_fullsized":"2024-09-22T03:30:41.914682+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.824965+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:41.824965+0000","last_peered":"2024-09-22T03:30:41.824965+0000","last_clean":"2024-09-22T03:30:41.824965+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:41.824965+0000","last_undegraded":"2024-09-22T03:30:41.824965+0000","last_fullsized":"2024-09-22T03:30:41.824965+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"71'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.828047+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:30:41.828047+0000","last_peered":"2024-09-22T03:30:41.828047+0000","last_clean":"2024-09-22T03:30:41.828047+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:30:41.828047+0000","last_undegraded":"2024-09-22T03:30:41.828047+0000","last_fullsized":"2024-09-22T03:30:41.828047+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":186,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.771937+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:41.771937+0000","last_peered":"2024-09-22T03:30:41.771937+0000","last_clean":"2024-09-22T03:30:41.771937+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:41.771937+0000","last_undegraded":"2024-09-22T03:30:41.771937+0000","last_fullsized":"2024-09-22T03:30:41.771937+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"70'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.774827+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:30:41.774827+0000","last_peered":"2024-09-22T03:30:41.774827+0000","last_clean":"2024-09-22T03:30:41.774827+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:30:41.774827+0000","last_undegraded":"2024-09-22T03:30:41.774827+0000","last_fullsized":"2024-09-22T03:30:41.774827+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819298+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:41.819298+0000","last_peered":"2024-09-22T03:30:41.819298+0000","last_clean":"2024-09-22T03:30:41.819298+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:41.819298+0000","last_undegraded":"2024-09-22T03:30:41.819298+0000","last_fullsized":"2024-09-22T03:30:41.819298+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"72'137","reported_seq":162,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.964776+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:30:41.964776+0000","last_peered":"2024-09-22T03:30:41.964776+0000","last_clean":"2024-09-22T03:30:41.964776+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:30:41.964776+0000","last_undegraded":"2024-09-22T03:30:41.964776+0000","last_fullsized":"2024-09-22T03:30:41.964776+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.821300+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:41.821300+0000","last_peered":"2024-09-22T03:30:41.821300+0000","last_clean":"2024-09-22T03:30:41.821300+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:41.821300+0000","last_undegraded":"2024-09-22T03:30:41.821300+0000","last_fullsized":"2024-09-22T03:30:41.821300+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"71'208","reported_seq":232,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.815560+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:30:41.815560+0000","last_peered":"2024-09-22T03:30:41.815560+0000","last_clean":"2024-09-22T03:30:41.815560+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:30:41.815560+0000","last_undegraded":"2024-09-22T03:30:41.815560+0000","last_fullsized":"2024-09-22T03:30:41.815560+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819859+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:41.819859+0000","last_peered":"2024-09-22T03:30:41.819859+0000","last_clean":"2024-09-22T03:30:41.819859+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:41.819859+0000","last_undegraded":"2024-09-22T03:30:41.819859+0000","last_fullsized":"2024-09-22T03:30:41.819859+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"70'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819727+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:30:41.819727+0000","last_peered":"2024-09-22T03:30:41.819727+0000","last_clean":"2024-09-22T03:30:41.819727+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:30:41.819727+0000","last_undegraded":"2024-09-22T03:30:41.819727+0000","last_fullsized":"2024-09-22T03:30:41.819727+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.828194+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:41.828194+0000","last_peered":"2024-09-22T03:30:41.828194+0000","last_clean":"2024-09-22T03:30:41.828194+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:41.828194+0000","last_undegraded":"2024-09-22T03:30:41.828194+0000","last_fullsized":"2024-09-22T03:30:41.828194+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":19152896,"num_objects":293,"num_object_clones":0,"num_object_copies":879,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":18704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214084112384,"internally_reserved":0,"allocated":127381504,"data_stored":127381504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2335,"ondisk_log_size":2335,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214084112384,"internally_reserved":0,"allocated":127381504,"data_stored":127381504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":37000,"kb_used_data":37000,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52260728,"statfs":{"total":53552873472,"available":53514985472,"internally_reserved":0,"allocated":37888000,"data_stored":37888000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23588,"kb_used_data":23588,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274140,"statfs":{"total":53552873472,"available":53528719360,"internally_reserved":0,"allocated":24154112,"data_stored":24154112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":31140,"kb_used_data":31140,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52266588,"statfs":{"total":53552873472,"available":53520986112,"internally_reserved":0,"allocated":31887360,"data_stored":31887360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":27272,"kb_used_data":27272,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52270456,"statfs":{"total":53552873472,"available":53524946944,"internally_reserved":0,"allocated":27926528,"data_stored":27926528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53523689472,"internally_reserved":0,"allocated":29184000,"data_stored":29184000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53520101376,"internally_reserved":0,"allocated":32772096,"data_stored":32772096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53527138304,"internally_reserved":0,"allocated":25735168,"data_stored":25735168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53523689472,"internally_reserved":0,"allocated":29184000,"data_stored":29184000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53520101376,"internally_reserved":0,"allocated":32772096,"data_stored":32772096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53527138304,"internally_reserved":0,"allocated":25735168,"data_stored":25735168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:30:45.108 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:30:45.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 2783 2767 1.13764 1.08594 0.0509678 0.109439 2024-09-22T03:30:45.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.218+0000 7fe949a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.226+0000 7fe949a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.423 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:30:45.424 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:30:45.438 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-22T03:30:44.769789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19743283,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":2477,"num_write_kb":20600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2435,"ondisk_log_size":2435,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":119000,"kb_used_data":119000,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209071912,"statfs":{"total":214211493888,"available":214089637888,"internally_reserved":0,"allocated":121856000,"data_stored":121856000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5791744,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":707,"num_write_kb":5656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040001"},"pg_stats":[{"pgid":"7.f","version":"71'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.816152+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:30:41.816152+0000","last_peered":"2024-09-22T03:30:41.816152+0000","last_clean":"2024-09-22T03:30:41.816152+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:30:41.816152+0000","last_undegraded":"2024-09-22T03:30:41.816152+0000","last_fullsized":"2024-09-22T03:30:41.816152+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"71'104","reported_seq":128,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.825573+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:30:41.825573+0000","last_peered":"2024-09-22T03:30:41.825573+0000","last_clean":"2024-09-22T03:30:41.825573+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:30:41.825573+0000","last_undegraded":"2024-09-22T03:30:41.825573+0000","last_fullsized":"2024-09-22T03:30:41.825573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"70'176","reported_seq":200,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.815013+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:30:41.815013+0000","last_peered":"2024-09-22T03:30:41.815013+0000","last_clean":"2024-09-22T03:30:41.815013+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:30:41.815013+0000","last_undegraded":"2024-09-22T03:30:41.815013+0000","last_fullsized":"2024-09-22T03:30:41.815013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"71'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819121+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:30:41.819121+0000","last_peered":"2024-09-22T03:30:41.819121+0000","last_clean":"2024-09-22T03:30:41.819121+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:30:41.819121+0000","last_undegraded":"2024-09-22T03:30:41.819121+0000","last_fullsized":"2024-09-22T03:30:41.819121+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"70'176","reported_seq":200,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.821854+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:30:41.821854+0000","last_peered":"2024-09-22T03:30:41.821854+0000","last_clean":"2024-09-22T03:30:41.821854+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:30:41.821854+0000","last_undegraded":"2024-09-22T03:30:41.821854+0000","last_fullsized":"2024-09-22T03:30:41.821854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"72'159","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.955212+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:30:41.955212+0000","last_peered":"2024-09-22T03:30:41.955212+0000","last_clean":"2024-09-22T03:30:41.955212+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:30:41.955212+0000","last_undegraded":"2024-09-22T03:30:41.955212+0000","last_fullsized":"2024-09-22T03:30:41.955212+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"70'192","reported_seq":216,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.816221+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:30:41.816221+0000","last_peered":"2024-09-22T03:30:41.816221+0000","last_clean":"2024-09-22T03:30:41.816221+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:30:41.816221+0000","last_undegraded":"2024-09-22T03:30:41.816221+0000","last_fullsized":"2024-09-22T03:30:41.816221+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"70'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.828404+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:30:41.828404+0000","last_peered":"2024-09-22T03:30:41.828404+0000","last_clean":"2024-09-22T03:30:41.828404+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:30:41.828404+0000","last_undegraded":"2024-09-22T03:30:41.828404+0000","last_fullsized":"2024-09-22T03:30:41.828404+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"70'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.820342+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:30:41.820342+0000","last_peered":"2024-09-22T03:30:41.820342+0000","last_clean":"2024-09-22T03:30:41.820342+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:30:41.820342+0000","last_undegraded":"2024-09-22T03:30:41.820342+0000","last_fullsized":"2024-09-22T03:30:41.820342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":319,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.817530+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:41.817530+0000","last_peered":"2024-09-22T03:30:41.817530+0000","last_clean":"2024-09-22T03:30:41.817530+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:41.817530+0000","last_undegraded":"2024-09-22T03:30:41.817530+0000","last_fullsized":"2024-09-22T03:30:41.817530+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":185,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.829327+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:41.829327+0000","last_peered":"2024-09-22T03:30:41.829327+0000","last_clean":"2024-09-22T03:30:41.829327+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:41.829327+0000","last_undegraded":"2024-09-22T03:30:41.829327+0000","last_fullsized":"2024-09-22T03:30:41.829327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"71'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.827259+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:30:41.827259+0000","last_peered":"2024-09-22T03:30:41.827259+0000","last_clean":"2024-09-22T03:30:41.827259+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:30:41.827259+0000","last_undegraded":"2024-09-22T03:30:41.827259+0000","last_fullsized":"2024-09-22T03:30:41.827259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.774204+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:41.774204+0000","last_peered":"2024-09-22T03:30:41.774204+0000","last_clean":"2024-09-22T03:30:41.774204+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:41.774204+0000","last_undegraded":"2024-09-22T03:30:41.774204+0000","last_fullsized":"2024-09-22T03:30:41.774204+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"72'127","reported_seq":152,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.914682+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:30:41.914682+0000","last_peered":"2024-09-22T03:30:41.914682+0000","last_clean":"2024-09-22T03:30:41.914682+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:30:41.914682+0000","last_undegraded":"2024-09-22T03:30:41.914682+0000","last_fullsized":"2024-09-22T03:30:41.914682+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.824965+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:41.824965+0000","last_peered":"2024-09-22T03:30:41.824965+0000","last_clean":"2024-09-22T03:30:41.824965+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:41.824965+0000","last_undegraded":"2024-09-22T03:30:41.824965+0000","last_fullsized":"2024-09-22T03:30:41.824965+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"71'120","reported_seq":144,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.828047+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:30:41.828047+0000","last_peered":"2024-09-22T03:30:41.828047+0000","last_clean":"2024-09-22T03:30:41.828047+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:30:41.828047+0000","last_undegraded":"2024-09-22T03:30:41.828047+0000","last_fullsized":"2024-09-22T03:30:41.828047+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":186,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.771937+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:41.771937+0000","last_peered":"2024-09-22T03:30:41.771937+0000","last_clean":"2024-09-22T03:30:41.771937+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:41.771937+0000","last_undegraded":"2024-09-22T03:30:41.771937+0000","last_fullsized":"2024-09-22T03:30:41.771937+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"70'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.774827+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:30:41.774827+0000","last_peered":"2024-09-22T03:30:41.774827+0000","last_clean":"2024-09-22T03:30:41.774827+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:30:41.774827+0000","last_undegraded":"2024-09-22T03:30:41.774827+0000","last_fullsized":"2024-09-22T03:30:41.774827+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819298+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:41.819298+0000","last_peered":"2024-09-22T03:30:41.819298+0000","last_clean":"2024-09-22T03:30:41.819298+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:41.819298+0000","last_undegraded":"2024-09-22T03:30:41.819298+0000","last_fullsized":"2024-09-22T03:30:41.819298+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"72'137","reported_seq":162,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.964776+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:30:41.964776+0000","last_peered":"2024-09-22T03:30:41.964776+0000","last_clean":"2024-09-22T03:30:41.964776+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:30:41.964776+0000","last_undegraded":"2024-09-22T03:30:41.964776+0000","last_fullsized":"2024-09-22T03:30:41.964776+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.821300+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:41.821300+0000","last_peered":"2024-09-22T03:30:41.821300+0000","last_clean":"2024-09-22T03:30:41.821300+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:41.821300+0000","last_undegraded":"2024-09-22T03:30:41.821300+0000","last_fullsized":"2024-09-22T03:30:41.821300+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"71'208","reported_seq":232,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.815560+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:30:41.815560+0000","last_peered":"2024-09-22T03:30:41.815560+0000","last_clean":"2024-09-22T03:30:41.815560+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:30:41.815560+0000","last_undegraded":"2024-09-22T03:30:41.815560+0000","last_fullsized":"2024-09-22T03:30:41.815560+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819859+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:41.819859+0000","last_peered":"2024-09-22T03:30:41.819859+0000","last_clean":"2024-09-22T03:30:41.819859+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:41.819859+0000","last_undegraded":"2024-09-22T03:30:41.819859+0000","last_fullsized":"2024-09-22T03:30:41.819859+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"70'144","reported_seq":168,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.819727+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:30:41.819727+0000","last_peered":"2024-09-22T03:30:41.819727+0000","last_clean":"2024-09-22T03:30:41.819727+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:30:41.819727+0000","last_undegraded":"2024-09-22T03:30:41.819727+0000","last_fullsized":"2024-09-22T03:30:41.819727+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":184,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T03:30:41.828194+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:41.828194+0000","last_peered":"2024-09-22T03:30:41.828194+0000","last_clean":"2024-09-22T03:30:41.828194+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:41.828194+0000","last_undegraded":"2024-09-22T03:30:41.828194+0000","last_fullsized":"2024-09-22T03:30:41.828194+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":19152896,"num_objects":293,"num_object_clones":0,"num_object_copies":879,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":18704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214084112384,"internally_reserved":0,"allocated":127381504,"data_stored":127381504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2335,"ondisk_log_size":2335,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214084112384,"internally_reserved":0,"allocated":127381504,"data_stored":127381504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":37000,"kb_used_data":37000,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52260728,"statfs":{"total":53552873472,"available":53514985472,"internally_reserved":0,"allocated":37888000,"data_stored":37888000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23588,"kb_used_data":23588,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274140,"statfs":{"total":53552873472,"available":53528719360,"internally_reserved":0,"allocated":24154112,"data_stored":24154112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":31140,"kb_used_data":31140,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52266588,"statfs":{"total":53552873472,"available":53520986112,"internally_reserved":0,"allocated":31887360,"data_stored":31887360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706123,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":27272,"kb_used_data":27272,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52270456,"statfs":{"total":53552873472,"available":53524946944,"internally_reserved":0,"allocated":27926528,"data_stored":27926528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53523689472,"internally_reserved":0,"allocated":29184000,"data_stored":29184000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53520101376,"internally_reserved":0,"allocated":32772096,"data_stored":32772096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53527138304,"internally_reserved":0,"allocated":25735168,"data_stored":25735168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53523689472,"internally_reserved":0,"allocated":29184000,"data_stored":29184000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53520101376,"internally_reserved":0,"allocated":32772096,"data_stored":32772096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53527138304,"internally_reserved":0,"allocated":25735168,"data_stored":25735168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53513183232,"internally_reserved":0,"allocated":39690240,"data_stored":39690240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:30:45.439 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:30:45.440 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:45.440 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:45.440 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:45.440 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:45.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.559+0000 7fdbd4f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.569+0000 7fdbd4f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.569+0000 7f2979a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.576+0000 7f2979a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.631 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:45.631 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:45.631 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:45.632 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:45.635 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:45.638 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:45.638 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:45.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.763+0000 7fcc1db57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.774+0000 7fcc1db57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.788 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:45.788 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:45.788 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:45.789 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:45.792 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:45.793 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:45.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:45.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.954+0000 7fb5f0b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:45.964+0000 7fb5f0b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:45.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:45.981 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:45.981 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:45.981 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:45.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:45.984 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:45.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:46.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:46.110+0000 7fd0be597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:46.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:46.119+0000 7fd0be597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:46.145 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:46.146 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:46.148 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:46.149 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:46.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:30:46.193223+0000 min lat: 0.0135712 max lat: 0.408893 avg lat: 0.109621 2024-09-22T03:30:46.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:30:46.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 2924 2908 1.13583 1.10156 0.265672 0.109621 2024-09-22T03:30:47.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 3091 3075 1.14387 1.30469 0.0239661 0.108702 2024-09-22T03:30:47.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:47.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:47.870+0000 7fc9b2997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:48.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:47.876+0000 7fc9b2997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:48.193 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 3249 3233 1.14798 1.23438 0.0825725 0.108555 2024-09-22T03:30:49.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 3407 3391 1.15173 1.23438 0.157632 0.107953 2024-09-22T03:30:50.156 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:50.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 3577 3561 1.15907 1.32812 0.0680264 0.107661 2024-09-22T03:30:50.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:50.279+0000 7f740544f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:50.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:50.284+0000 7f740544f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.149 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:51.150 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:51.150 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:30:51.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:30:51.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 3736 3720 1.16239 1.24219 0.0973763 0.107203 2024-09-22T03:30:51.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.274+0000 7f67942a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.286+0000 7f67942a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.319 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:51.319 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:51.319 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:51.319 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:51.319 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:51.320 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:51.323 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:51.323 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:51.323 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:30:51.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.487+0000 7f0b54517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.498+0000 7f0b54517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:51.515 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:51.516 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:51.516 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:51.516 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:51.519 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:51.519 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:51.519 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:30:51.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.633+0000 7f63bc50e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.643+0000 7f63bc50e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.693 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:51.693 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:51.693 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:51.693 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:51.694 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:51.694 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:51.694 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:51.694 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:51.694 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:51.694 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:51.694 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:51.695 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:51.696 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:51.697 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:51.697 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:30:51.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.815+0000 7f17f1863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:51.824+0000 7f17f1863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:51.902 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:30:51.902 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:30:51.902 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:30:51.902 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:30:51.902 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:30:51.902 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:30:51.902 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:30:51.903 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:30:51.903 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:30:51.903 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:30:51.903 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:30:51.903 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:30:51.904 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:30:51.905 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:30:52.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 3889 3873 1.16365 1.19531 0.176105 0.107037 2024-09-22T03:30:52.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:52.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:52.640+0000 7f1ddff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:52.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:52.646+0000 7f1ddff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:53.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 4024 4008 1.15961 1.05469 0.0606704 0.107468 2024-09-22T03:30:54.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 4173 4157 1.15977 1.16406 0.132919 0.107537 2024-09-22T03:30:54.797 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:54.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:54.926+0000 7f747a10e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:54.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:54.932+0000 7f747a10e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:55.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 4320 4304 1.15937 1.14844 0.0657979 0.107609 2024-09-22T03:30:56.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 4466 4450 1.15875 1.14062 0.194252 0.107557 2024-09-22T03:30:56.906 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:30:56.906 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:30:56.906 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:30:57.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:57.015+0000 7fd200cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:57.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:57.022+0000 7fd200cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:57.092 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:30:57.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 4635 4619 1.16395 1.32031 0.150181 0.107134 2024-09-22T03:30:57.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:57.222+0000 7fda3a597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:57.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:57.230+0000 7fda3a597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:57.253 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:30:57.253 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:30:57.278 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-22T03:30:56.022427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32154163,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3992,"num_write_kb":32720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3937,"ondisk_log_size":3937,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":192016,"kb_used_data":192016,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208998896,"statfs":{"total":214211493888,"available":214014869504,"internally_reserved":0,"allocated":196624384,"data_stored":196624384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12410880,"num_objects":189,"num_object_clones":0,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1515,"num_write_kb":12120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.289780"},"pg_stats":[{"pgid":"7.f","version":"76'231","reported_seq":268,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:49.989385+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:30:49.989385+0000","last_peered":"2024-09-22T03:30:49.989385+0000","last_clean":"2024-09-22T03:30:49.989385+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:30:49.989385+0000","last_undegraded":"2024-09-22T03:30:49.989385+0000","last_fullsized":"2024-09-22T03:30:49.989385+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'175","reported_seq":212,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:50.761788+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:30:50.761788+0000","last_peered":"2024-09-22T03:30:50.761788+0000","last_clean":"2024-09-22T03:30:50.761788+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:30:50.761788+0000","last_undegraded":"2024-09-22T03:30:50.761788+0000","last_fullsized":"2024-09-22T03:30:50.761788+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'279","reported_seq":316,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.604525+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:30:51.604525+0000","last_peered":"2024-09-22T03:30:51.604525+0000","last_clean":"2024-09-22T03:30:51.604525+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:30:51.604525+0000","last_undegraded":"2024-09-22T03:30:51.604525+0000","last_fullsized":"2024-09-22T03:30:51.604525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'223","reported_seq":260,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:50.314957+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:30:50.314957+0000","last_peered":"2024-09-22T03:30:50.314957+0000","last_clean":"2024-09-22T03:30:50.314957+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:30:50.314957+0000","last_undegraded":"2024-09-22T03:30:50.314957+0000","last_fullsized":"2024-09-22T03:30:50.314957+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'255","reported_seq":292,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.758322+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:30:51.758322+0000","last_peered":"2024-09-22T03:30:51.758322+0000","last_clean":"2024-09-22T03:30:51.758322+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:30:51.758322+0000","last_undegraded":"2024-09-22T03:30:51.758322+0000","last_fullsized":"2024-09-22T03:30:51.758322+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'295","reported_seq":332,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.129606+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:30:51.129606+0000","last_peered":"2024-09-22T03:30:51.129606+0000","last_clean":"2024-09-22T03:30:51.129606+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:30:51.129606+0000","last_undegraded":"2024-09-22T03:30:51.129606+0000","last_fullsized":"2024-09-22T03:30:51.129606+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'271","reported_seq":308,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.857833+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:30:51.857833+0000","last_peered":"2024-09-22T03:30:51.857833+0000","last_clean":"2024-09-22T03:30:51.857833+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:30:51.857833+0000","last_undegraded":"2024-09-22T03:30:51.857833+0000","last_fullsized":"2024-09-22T03:30:51.857833+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'223","reported_seq":260,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.430553+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:30:51.430553+0000","last_peered":"2024-09-22T03:30:51.430553+0000","last_clean":"2024-09-22T03:30:51.430553+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:30:51.430553+0000","last_undegraded":"2024-09-22T03:30:51.430553+0000","last_fullsized":"2024-09-22T03:30:51.430553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'199","reported_seq":236,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:49.622684+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:30:49.622684+0000","last_peered":"2024-09-22T03:30:49.622684+0000","last_clean":"2024-09-22T03:30:49.622684+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:30:49.622684+0000","last_undegraded":"2024-09-22T03:30:49.622684+0000","last_fullsized":"2024-09-22T03:30:49.622684+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'271","reported_seq":308,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.660511+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:30:51.660511+0000","last_peered":"2024-09-22T03:30:51.660511+0000","last_clean":"2024-09-22T03:30:51.660511+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:30:51.660511+0000","last_undegraded":"2024-09-22T03:30:51.660511+0000","last_fullsized":"2024-09-22T03:30:51.660511+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'223","reported_seq":260,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.623450+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:30:51.623450+0000","last_peered":"2024-09-22T03:30:51.623450+0000","last_clean":"2024-09-22T03:30:51.623450+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:30:51.623450+0000","last_undegraded":"2024-09-22T03:30:51.623450+0000","last_fullsized":"2024-09-22T03:30:51.623450+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'199","reported_seq":236,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.943968+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:30:51.943968+0000","last_peered":"2024-09-22T03:30:51.943968+0000","last_clean":"2024-09-22T03:30:51.943968+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:30:51.943968+0000","last_undegraded":"2024-09-22T03:30:51.943968+0000","last_fullsized":"2024-09-22T03:30:51.943968+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'236","reported_seq":273,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.969489+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:30:51.969489+0000","last_peered":"2024-09-22T03:30:51.969489+0000","last_clean":"2024-09-22T03:30:51.969489+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:30:51.969489+0000","last_undegraded":"2024-09-22T03:30:51.969489+0000","last_fullsized":"2024-09-22T03:30:51.969489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1941504,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":237,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'223","reported_seq":260,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.803183+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:30:51.803183+0000","last_peered":"2024-09-22T03:30:51.803183+0000","last_clean":"2024-09-22T03:30:51.803183+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:30:51.803183+0000","last_undegraded":"2024-09-22T03:30:51.803183+0000","last_fullsized":"2024-09-22T03:30:51.803183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'311","reported_seq":348,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:51.705341+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:30:51.705341+0000","last_peered":"2024-09-22T03:30:51.705341+0000","last_clean":"2024-09-22T03:30:51.705341+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:30:51.705341+0000","last_undegraded":"2024-09-22T03:30:51.705341+0000","last_fullsized":"2024-09-22T03:30:51.705341+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'223","reported_seq":260,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:50.659948+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:30:50.659948+0000","last_peered":"2024-09-22T03:30:50.659948+0000","last_clean":"2024-09-22T03:30:50.659948+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:30:50.659948+0000","last_undegraded":"2024-09-22T03:30:50.659948+0000","last_fullsized":"2024-09-22T03:30:50.659948+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":31563776,"num_objects":482,"num_object_clones":0,"num_object_copies":1446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":30824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214007750656,"internally_reserved":0,"allocated":203743232,"data_stored":203743232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3837,"ondisk_log_size":3837,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214007750656,"internally_reserved":0,"allocated":203743232,"data_stored":203743232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53488783360,"internally_reserved":0,"allocated":64090112,"data_stored":64090112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706133,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":60196,"kb_used_data":60196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52237532,"statfs":{"total":53552873472,"available":53491232768,"internally_reserved":0,"allocated":61640704,"data_stored":61640704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706133,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":38492,"kb_used_data":38492,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52259236,"statfs":{"total":53552873472,"available":53513457664,"internally_reserved":0,"allocated":39415808,"data_stored":39415808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706133,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":49864,"kb_used_data":49864,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52247864,"statfs":{"total":53552873472,"available":53501812736,"internally_reserved":0,"allocated":51060736,"data_stored":51060736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706133,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":43464,"kb_used_data":43464,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52254264,"statfs":{"total":53552873472,"available":53508366336,"internally_reserved":0,"allocated":44507136,"data_stored":44507136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53488783360,"internally_reserved":0,"allocated":64090112,"data_stored":64090112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53506838528,"internally_reserved":0,"allocated":46034944,"data_stored":46034944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53500289024,"internally_reserved":0,"allocated":52584448,"data_stored":52584448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53511847936,"internally_reserved":0,"allocated":41025536,"data_stored":41025536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53488775168,"internally_reserved":0,"allocated":64098304,"data_stored":64098304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53506838528,"internally_reserved":0,"allocated":46034944,"data_stored":46034944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53500289024,"internally_reserved":0,"allocated":52584448,"data_stored":52584448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53511847936,"internally_reserved":0,"allocated":41025536,"data_stored":41025536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53488775168,"internally_reserved":0,"allocated":64098304,"data_stored":64098304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:30:58.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 16 4786 4770 1.16444 1.17969 0.0732877 0.107191 2024-09-22T03:30:59.194 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 4945 4929 1.1668 1.24219 0.150685 0.10689 2024-09-22T03:30:59.410 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:30:59.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:59.552+0000 7fb4ab357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:30:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:30:59.558+0000 7fb4ab357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:00.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 5091 5075 1.16602 1.14062 0.136479 0.10673 2024-09-22T03:31:01.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 5265 5249 1.17154 1.35938 0.141914 0.106499 2024-09-22T03:31:01.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:01.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:01.863+0000 7fae38797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:01.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:01.869+0000 7fae38797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:02.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 5407 5391 1.16981 1.10938 0.0705296 0.10665 2024-09-22T03:31:02.280 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:02.280 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:02.280 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:02.398+0000 7f7c33e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:02.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:02.406+0000 7f7c33e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:02.633 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:31:02.634 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:31:02.648 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-22T03:31:02.023584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":44573235,"num_objects":676,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5508,"num_write_kb":44848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5453,"ondisk_log_size":5453,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":269652,"kb_used_data":269652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208921260,"statfs":{"total":214211493888,"available":213935370240,"internally_reserved":0,"allocated":276123648,"data_stored":276123648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18907136,"num_objects":289,"num_object_clones":0,"num_object_copies":867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2308,"num_write_kb":18464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002301"},"pg_stats":[{"pgid":"7.f","version":"76'351","reported_seq":388,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.953813+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:31:01.953813+0000","last_peered":"2024-09-22T03:31:01.953813+0000","last_clean":"2024-09-22T03:31:01.953813+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:31:01.953813+0000","last_undegraded":"2024-09-22T03:31:01.953813+0000","last_fullsized":"2024-09-22T03:31:01.953813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'263","reported_seq":300,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.427012+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:31:00.427012+0000","last_peered":"2024-09-22T03:31:00.427012+0000","last_clean":"2024-09-22T03:31:00.427012+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:31:00.427012+0000","last_undegraded":"2024-09-22T03:31:00.427012+0000","last_fullsized":"2024-09-22T03:31:00.427012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'367","reported_seq":404,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.958818+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:31:00.958818+0000","last_peered":"2024-09-22T03:31:00.958818+0000","last_clean":"2024-09-22T03:31:00.958818+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:31:00.958818+0000","last_undegraded":"2024-09-22T03:31:00.958818+0000","last_fullsized":"2024-09-22T03:31:00.958818+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'319","reported_seq":356,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.343682+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:31:00.343682+0000","last_peered":"2024-09-22T03:31:00.343682+0000","last_clean":"2024-09-22T03:31:00.343682+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:31:00.343682+0000","last_undegraded":"2024-09-22T03:31:00.343682+0000","last_fullsized":"2024-09-22T03:31:00.343682+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'343","reported_seq":380,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.836060+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:31:00.836060+0000","last_peered":"2024-09-22T03:31:00.836060+0000","last_clean":"2024-09-22T03:31:00.836060+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:31:00.836060+0000","last_undegraded":"2024-09-22T03:31:00.836060+0000","last_fullsized":"2024-09-22T03:31:00.836060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'423","reported_seq":460,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.660994+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:31:01.660994+0000","last_peered":"2024-09-22T03:31:01.660994+0000","last_clean":"2024-09-22T03:31:01.660994+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:31:01.660994+0000","last_undegraded":"2024-09-22T03:31:01.660994+0000","last_fullsized":"2024-09-22T03:31:01.660994+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'399","reported_seq":436,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.898282+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:31:01.898282+0000","last_peered":"2024-09-22T03:31:01.898282+0000","last_clean":"2024-09-22T03:31:01.898282+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:31:01.898282+0000","last_undegraded":"2024-09-22T03:31:01.898282+0000","last_fullsized":"2024-09-22T03:31:01.898282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'287","reported_seq":324,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.853432+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:31:01.853432+0000","last_peered":"2024-09-22T03:31:01.853432+0000","last_clean":"2024-09-22T03:31:01.853432+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:31:01.853432+0000","last_undegraded":"2024-09-22T03:31:01.853432+0000","last_fullsized":"2024-09-22T03:31:01.853432+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'279","reported_seq":316,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:58.950218+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:30:58.950218+0000","last_peered":"2024-09-22T03:30:58.950218+0000","last_clean":"2024-09-22T03:30:58.950218+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:30:58.950218+0000","last_undegraded":"2024-09-22T03:30:58.950218+0000","last_fullsized":"2024-09-22T03:30:58.950218+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'375","reported_seq":412,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.856160+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:31:00.856160+0000","last_peered":"2024-09-22T03:31:00.856160+0000","last_clean":"2024-09-22T03:31:00.856160+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:31:00.856160+0000","last_undegraded":"2024-09-22T03:31:00.856160+0000","last_fullsized":"2024-09-22T03:31:00.856160+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'271","reported_seq":308,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:57.817217+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:30:57.817217+0000","last_peered":"2024-09-22T03:30:57.817217+0000","last_clean":"2024-09-22T03:30:57.817217+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:30:57.817217+0000","last_undegraded":"2024-09-22T03:30:57.817217+0000","last_fullsized":"2024-09-22T03:30:57.817217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'287","reported_seq":324,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.346804+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:31:01.346804+0000","last_peered":"2024-09-22T03:31:01.346804+0000","last_clean":"2024-09-22T03:31:01.346804+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:31:01.346804+0000","last_undegraded":"2024-09-22T03:31:01.346804+0000","last_fullsized":"2024-09-22T03:31:01.346804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'335","reported_seq":372,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.143127+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:31:01.143127+0000","last_peered":"2024-09-22T03:31:01.143127+0000","last_clean":"2024-09-22T03:31:01.143127+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:31:01.143127+0000","last_undegraded":"2024-09-22T03:31:01.143127+0000","last_fullsized":"2024-09-22T03:31:01.143127+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'279","reported_seq":316,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.661138+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:31:01.661138+0000","last_peered":"2024-09-22T03:31:01.661138+0000","last_clean":"2024-09-22T03:31:01.661138+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:31:01.661138+0000","last_undegraded":"2024-09-22T03:31:01.661138+0000","last_fullsized":"2024-09-22T03:31:01.661138+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'455","reported_seq":492,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.765464+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:31:00.765464+0000","last_peered":"2024-09-22T03:31:00.765464+0000","last_clean":"2024-09-22T03:31:00.765464+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:31:00.765464+0000","last_undegraded":"2024-09-22T03:31:00.765464+0000","last_fullsized":"2024-09-22T03:31:00.765464+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'320","reported_seq":357,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.508003+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:31:01.508003+0000","last_peered":"2024-09-22T03:31:01.508003+0000","last_clean":"2024-09-22T03:31:01.508003+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:31:01.508003+0000","last_undegraded":"2024-09-22T03:31:01.508003+0000","last_fullsized":"2024-09-22T03:31:01.508003+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2629632,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":321,"num_write_kb":2568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":43982848,"num_objects":672,"num_object_clones":0,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5369,"num_write_kb":42952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213929299968,"internally_reserved":0,"allocated":282193920,"data_stored":282193920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5353,"ondisk_log_size":5353,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213929316352,"internally_reserved":0,"allocated":282177536,"data_stored":282177536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":84516,"kb_used_data":84516,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52213212,"statfs":{"total":53552873472,"available":53466329088,"internally_reserved":0,"allocated":86544384,"data_stored":86544384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":54020,"kb_used_data":54020,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52243708,"statfs":{"total":53552873472,"available":53497556992,"internally_reserved":0,"allocated":55316480,"data_stored":55316480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":70052,"kb_used_data":70052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52227676,"statfs":{"total":53552873472,"available":53481140224,"internally_reserved":0,"allocated":71733248,"data_stored":71733248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":61064,"kb_used_data":61064,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52236664,"statfs":{"total":53552873472,"available":53490343936,"internally_reserved":0,"allocated":62529536,"data_stored":62529536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53488865280,"internally_reserved":0,"allocated":64008192,"data_stored":64008192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53479612416,"internally_reserved":0,"allocated":73261056,"data_stored":73261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53496631296,"internally_reserved":0,"allocated":56242176,"data_stored":56242176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53488865280,"internally_reserved":0,"allocated":64008192,"data_stored":64008192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53479596032,"internally_reserved":0,"allocated":73277440,"data_stored":73277440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53496631296,"internally_reserved":0,"allocated":56242176,"data_stored":56242176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:03.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 5552 5536 1.16881 1.13281 0.235286 0.106653 2024-09-22T03:31:04.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:04.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:04.139+0000 7fa5ac66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:04.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:04.145+0000 7fa5ac66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:04.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 5700 5684 1.16848 1.15625 0.0455421 0.106769 2024-09-22T03:31:05.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 5859 5843 1.17036 1.24219 0.146438 0.10665 2024-09-22T03:31:06.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:31:06.195072+0000 min lat: 0.0135712 max lat: 0.431193 avg lat: 0.106316 2024-09-22T03:31:06.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:31:06.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 6024 6008 1.17333 1.28906 0.125275 0.106316 2024-09-22T03:31:06.401 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:06.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:06.514+0000 7f3ecb8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:06.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:06.520+0000 7f3ecb8d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:07.195 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 6162 6146 1.171 1.07812 0.123116 0.106556 2024-09-22T03:31:07.650 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:07.650 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:07.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:07.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:07.766+0000 7f7e85d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:07.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:07.775+0000 7f7e85d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:07.976 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:31:07.976 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:31:08.001 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-22T03:31:06.024341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":44573235,"num_objects":676,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5508,"num_write_kb":44848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5453,"ondisk_log_size":5453,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":269652,"kb_used_data":269652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208921260,"statfs":{"total":214211493888,"available":213935370240,"internally_reserved":0,"allocated":276123648,"data_stored":276123648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12419072,"num_objects":190,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1516,"num_write_kb":12128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002265"},"pg_stats":[{"pgid":"7.f","version":"76'351","reported_seq":388,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.953813+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:31:01.953813+0000","last_peered":"2024-09-22T03:31:01.953813+0000","last_clean":"2024-09-22T03:31:01.953813+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:31:01.953813+0000","last_undegraded":"2024-09-22T03:31:01.953813+0000","last_fullsized":"2024-09-22T03:31:01.953813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'263","reported_seq":300,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.427012+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:31:00.427012+0000","last_peered":"2024-09-22T03:31:00.427012+0000","last_clean":"2024-09-22T03:31:00.427012+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:31:00.427012+0000","last_undegraded":"2024-09-22T03:31:00.427012+0000","last_fullsized":"2024-09-22T03:31:00.427012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'367","reported_seq":404,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.958818+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:31:00.958818+0000","last_peered":"2024-09-22T03:31:00.958818+0000","last_clean":"2024-09-22T03:31:00.958818+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:31:00.958818+0000","last_undegraded":"2024-09-22T03:31:00.958818+0000","last_fullsized":"2024-09-22T03:31:00.958818+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'319","reported_seq":356,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.343682+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:31:00.343682+0000","last_peered":"2024-09-22T03:31:00.343682+0000","last_clean":"2024-09-22T03:31:00.343682+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:31:00.343682+0000","last_undegraded":"2024-09-22T03:31:00.343682+0000","last_fullsized":"2024-09-22T03:31:00.343682+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'343","reported_seq":380,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.836060+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:31:00.836060+0000","last_peered":"2024-09-22T03:31:00.836060+0000","last_clean":"2024-09-22T03:31:00.836060+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:31:00.836060+0000","last_undegraded":"2024-09-22T03:31:00.836060+0000","last_fullsized":"2024-09-22T03:31:00.836060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'423","reported_seq":460,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.660994+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:31:01.660994+0000","last_peered":"2024-09-22T03:31:01.660994+0000","last_clean":"2024-09-22T03:31:01.660994+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:31:01.660994+0000","last_undegraded":"2024-09-22T03:31:01.660994+0000","last_fullsized":"2024-09-22T03:31:01.660994+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'399","reported_seq":436,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.898282+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:31:01.898282+0000","last_peered":"2024-09-22T03:31:01.898282+0000","last_clean":"2024-09-22T03:31:01.898282+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:31:01.898282+0000","last_undegraded":"2024-09-22T03:31:01.898282+0000","last_fullsized":"2024-09-22T03:31:01.898282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'287","reported_seq":324,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.853432+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:31:01.853432+0000","last_peered":"2024-09-22T03:31:01.853432+0000","last_clean":"2024-09-22T03:31:01.853432+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:31:01.853432+0000","last_undegraded":"2024-09-22T03:31:01.853432+0000","last_fullsized":"2024-09-22T03:31:01.853432+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'279","reported_seq":316,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:58.950218+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:30:58.950218+0000","last_peered":"2024-09-22T03:30:58.950218+0000","last_clean":"2024-09-22T03:30:58.950218+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:30:58.950218+0000","last_undegraded":"2024-09-22T03:30:58.950218+0000","last_fullsized":"2024-09-22T03:30:58.950218+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'375","reported_seq":412,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.856160+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:31:00.856160+0000","last_peered":"2024-09-22T03:31:00.856160+0000","last_clean":"2024-09-22T03:31:00.856160+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:31:00.856160+0000","last_undegraded":"2024-09-22T03:31:00.856160+0000","last_fullsized":"2024-09-22T03:31:00.856160+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'271","reported_seq":308,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:57.817217+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:30:57.817217+0000","last_peered":"2024-09-22T03:30:57.817217+0000","last_clean":"2024-09-22T03:30:57.817217+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:30:57.817217+0000","last_undegraded":"2024-09-22T03:30:57.817217+0000","last_fullsized":"2024-09-22T03:30:57.817217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'287","reported_seq":324,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.346804+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:31:01.346804+0000","last_peered":"2024-09-22T03:31:01.346804+0000","last_clean":"2024-09-22T03:31:01.346804+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:31:01.346804+0000","last_undegraded":"2024-09-22T03:31:01.346804+0000","last_fullsized":"2024-09-22T03:31:01.346804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'335","reported_seq":372,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.143127+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:31:01.143127+0000","last_peered":"2024-09-22T03:31:01.143127+0000","last_clean":"2024-09-22T03:31:01.143127+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:31:01.143127+0000","last_undegraded":"2024-09-22T03:31:01.143127+0000","last_fullsized":"2024-09-22T03:31:01.143127+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'279","reported_seq":316,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.661138+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:31:01.661138+0000","last_peered":"2024-09-22T03:31:01.661138+0000","last_clean":"2024-09-22T03:31:01.661138+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:31:01.661138+0000","last_undegraded":"2024-09-22T03:31:01.661138+0000","last_fullsized":"2024-09-22T03:31:01.661138+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'455","reported_seq":492,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:00.765464+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:31:00.765464+0000","last_peered":"2024-09-22T03:31:00.765464+0000","last_clean":"2024-09-22T03:31:00.765464+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:31:00.765464+0000","last_undegraded":"2024-09-22T03:31:00.765464+0000","last_fullsized":"2024-09-22T03:31:00.765464+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'320","reported_seq":357,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:01.508003+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:31:01.508003+0000","last_peered":"2024-09-22T03:31:01.508003+0000","last_clean":"2024-09-22T03:31:01.508003+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:31:01.508003+0000","last_undegraded":"2024-09-22T03:31:01.508003+0000","last_fullsized":"2024-09-22T03:31:01.508003+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2629632,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":321,"num_write_kb":2568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":43982848,"num_objects":672,"num_object_clones":0,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5369,"num_write_kb":42952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213929299968,"internally_reserved":0,"allocated":282193920,"data_stored":282193920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5353,"ondisk_log_size":5353,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213929316352,"internally_reserved":0,"allocated":282177536,"data_stored":282177536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":84516,"kb_used_data":84516,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52213212,"statfs":{"total":53552873472,"available":53466329088,"internally_reserved":0,"allocated":86544384,"data_stored":86544384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":54020,"kb_used_data":54020,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52243708,"statfs":{"total":53552873472,"available":53497556992,"internally_reserved":0,"allocated":55316480,"data_stored":55316480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":70052,"kb_used_data":70052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52227676,"statfs":{"total":53552873472,"available":53481140224,"internally_reserved":0,"allocated":71733248,"data_stored":71733248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706143,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":61064,"kb_used_data":61064,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52236664,"statfs":{"total":53552873472,"available":53490343936,"internally_reserved":0,"allocated":62529536,"data_stored":62529536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53488865280,"internally_reserved":0,"allocated":64008192,"data_stored":64008192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53479612416,"internally_reserved":0,"allocated":73261056,"data_stored":73261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53496631296,"internally_reserved":0,"allocated":56242176,"data_stored":56242176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53488865280,"internally_reserved":0,"allocated":64008192,"data_stored":64008192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53479596032,"internally_reserved":0,"allocated":73277440,"data_stored":73277440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53496631296,"internally_reserved":0,"allocated":56242176,"data_stored":56242176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53464207360,"internally_reserved":0,"allocated":88666112,"data_stored":88666112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:08.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 6316 6300 1.17177 1.20312 0.0610743 0.106474 2024-09-22T03:31:08.675 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:08.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:08.782+0000 7f8a02b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:08.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:08.787+0000 7f8a02b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:09.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 16 6456 6440 1.16995 1.09375 0.173415 0.10668 2024-09-22T03:31:10.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 6605 6589 1.16981 1.16406 0.0934755 0.106718 2024-09-22T03:31:11.017 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:11.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:11.126+0000 7fd43486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:11.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:11.131+0000 7fd43486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:11.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 6749 6733 1.16881 1.125 0.259312 0.106679 2024-09-22T03:31:12.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 6901 6885 1.16922 1.1875 0.119671 0.106794 2024-09-22T03:31:13.003 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:13.003 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:13.003 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:31:13.004 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:31:13.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.117+0000 7f51d54d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.127+0000 7f51d54d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 15 7071 7056 1.17276 1.33594 0.0809419 0.106497 2024-09-22T03:31:13.213 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:13.214 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:13.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:13.218 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:13.218 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:31:13.359 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:13.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.379+0000 7fd95d4ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.393+0000 7fd95d4ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.438 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:13.438 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:13.438 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:13.438 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:13.442 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:13.442 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:13.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:31:13.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.494+0000 7fd36906c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.501+0000 7fd36906c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.624+0000 7f8c4fd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.633+0000 7f8c4fd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.649 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:13.649 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:13.649 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:13.650 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:13.653 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:13.653 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:13.653 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:31:13.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.868+0000 7f3a26717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:13.881+0000 7f3a26717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:13.968 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:13.968 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:13.968 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:13.968 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:13.968 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:13.968 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:13.969 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:13.969 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:13.969 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:13.969 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:13.969 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:13.969 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:13.971 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:13.971 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:14.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 7216 7200 1.17176 1.125 0.0990418 0.106549 2024-09-22T03:31:15.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 7367 7351 1.17192 1.17969 0.0546298 0.106451 2024-09-22T03:31:15.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:15.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:15.797+0000 7f313ff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:15.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:15.803+0000 7f313ff17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:16.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 7505 7489 1.17005 1.07812 0.166867 0.106684 2024-09-22T03:31:17.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 7656 7640 1.17023 1.17969 0.20293 0.106658 2024-09-22T03:31:18.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:18.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:18.146+0000 7fa784b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:18.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:18.152+0000 7fa784b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:18.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 7805 7789 1.17011 1.16406 0.0963464 0.10668 2024-09-22T03:31:18.972 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:18.973 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:18.973 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:19.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:19.117+0000 7fbff4a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:19.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:19.128+0000 7fbff4a18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:19.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 7948 7932 1.16911 1.11719 0.129148 0.10676 2024-09-22T03:31:19.353 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:31:19.354 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:31:19.369 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-22T03:31:18.026549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":63054387,"num_objects":958,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":7764,"num_write_kb":62896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7709,"ondisk_log_size":7709,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":381512,"kb_used_data":381512,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208809400,"statfs":{"total":214211493888,"available":213820825600,"internally_reserved":0,"allocated":390668288,"data_stored":390668288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18481152,"num_objects":282,"num_object_clones":0,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002208"},"pg_stats":[{"pgid":"7.f","version":"76'463","reported_seq":500,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.519473+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:31:16.519473+0000","last_peered":"2024-09-22T03:31:16.519473+0000","last_clean":"2024-09-22T03:31:16.519473+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:31:16.519473+0000","last_undegraded":"2024-09-22T03:31:16.519473+0000","last_fullsized":"2024-09-22T03:31:16.519473+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'383","reported_seq":420,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:15.850823+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:31:15.850823+0000","last_peered":"2024-09-22T03:31:15.850823+0000","last_clean":"2024-09-22T03:31:15.850823+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:31:15.850823+0000","last_undegraded":"2024-09-22T03:31:15.850823+0000","last_fullsized":"2024-09-22T03:31:15.850823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'527","reported_seq":564,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:14.682622+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:31:14.682622+0000","last_peered":"2024-09-22T03:31:14.682622+0000","last_clean":"2024-09-22T03:31:14.682622+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:31:14.682622+0000","last_undegraded":"2024-09-22T03:31:14.682622+0000","last_fullsized":"2024-09-22T03:31:14.682622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'423","reported_seq":460,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.361449+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:31:16.361449+0000","last_peered":"2024-09-22T03:31:16.361449+0000","last_clean":"2024-09-22T03:31:16.361449+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:31:16.361449+0000","last_undegraded":"2024-09-22T03:31:16.361449+0000","last_fullsized":"2024-09-22T03:31:16.361449+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'503","reported_seq":540,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:15.565526+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:31:15.565526+0000","last_peered":"2024-09-22T03:31:15.565526+0000","last_clean":"2024-09-22T03:31:15.565526+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:31:15.565526+0000","last_undegraded":"2024-09-22T03:31:15.565526+0000","last_fullsized":"2024-09-22T03:31:15.565526+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'583","reported_seq":620,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.855823+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:31:16.855823+0000","last_peered":"2024-09-22T03:31:16.855823+0000","last_clean":"2024-09-22T03:31:16.855823+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:31:16.855823+0000","last_undegraded":"2024-09-22T03:31:16.855823+0000","last_fullsized":"2024-09-22T03:31:16.855823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'535","reported_seq":572,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.622738+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:31:16.622738+0000","last_peered":"2024-09-22T03:31:16.622738+0000","last_clean":"2024-09-22T03:31:16.622738+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:31:16.622738+0000","last_undegraded":"2024-09-22T03:31:16.622738+0000","last_fullsized":"2024-09-22T03:31:16.622738+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'375","reported_seq":412,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:14.951900+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:31:14.951900+0000","last_peered":"2024-09-22T03:31:14.951900+0000","last_clean":"2024-09-22T03:31:14.951900+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:31:14.951900+0000","last_undegraded":"2024-09-22T03:31:14.951900+0000","last_fullsized":"2024-09-22T03:31:14.951900+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'423","reported_seq":460,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.789544+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:31:16.789544+0000","last_peered":"2024-09-22T03:31:16.789544+0000","last_clean":"2024-09-22T03:31:16.789544+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:31:16.789544+0000","last_undegraded":"2024-09-22T03:31:16.789544+0000","last_fullsized":"2024-09-22T03:31:16.789544+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'535","reported_seq":572,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:15.010067+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:31:15.010067+0000","last_peered":"2024-09-22T03:31:15.010067+0000","last_clean":"2024-09-22T03:31:15.010067+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:31:15.010067+0000","last_undegraded":"2024-09-22T03:31:15.010067+0000","last_fullsized":"2024-09-22T03:31:15.010067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'408","reported_seq":445,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.910967+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:31:16.910967+0000","last_peered":"2024-09-22T03:31:16.910967+0000","last_clean":"2024-09-22T03:31:16.910967+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:31:16.910967+0000","last_undegraded":"2024-09-22T03:31:16.910967+0000","last_fullsized":"2024-09-22T03:31:16.910967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3350528,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":409,"num_write_kb":3272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'447","reported_seq":484,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.031613+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:31:16.031613+0000","last_peered":"2024-09-22T03:31:16.031613+0000","last_clean":"2024-09-22T03:31:16.031613+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:31:16.031613+0000","last_undegraded":"2024-09-22T03:31:16.031613+0000","last_fullsized":"2024-09-22T03:31:16.031613+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'511","reported_seq":548,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.186411+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:31:16.186411+0000","last_peered":"2024-09-22T03:31:16.186411+0000","last_clean":"2024-09-22T03:31:16.186411+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:31:16.186411+0000","last_undegraded":"2024-09-22T03:31:16.186411+0000","last_fullsized":"2024-09-22T03:31:16.186411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'407","reported_seq":444,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.678664+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:31:16.678664+0000","last_peered":"2024-09-22T03:31:16.678664+0000","last_clean":"2024-09-22T03:31:16.678664+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:31:16.678664+0000","last_undegraded":"2024-09-22T03:31:16.678664+0000","last_fullsized":"2024-09-22T03:31:16.678664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'607","reported_seq":644,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.732347+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:31:16.732347+0000","last_peered":"2024-09-22T03:31:16.732347+0000","last_clean":"2024-09-22T03:31:16.732347+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:31:16.732347+0000","last_undegraded":"2024-09-22T03:31:16.732347+0000","last_fullsized":"2024-09-22T03:31:16.732347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'479","reported_seq":516,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:16.953840+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:31:16.953840+0000","last_peered":"2024-09-22T03:31:16.953840+0000","last_clean":"2024-09-22T03:31:16.953840+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:31:16.953840+0000","last_undegraded":"2024-09-22T03:31:16.953840+0000","last_fullsized":"2024-09-22T03:31:16.953840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":62464000,"num_objects":954,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7625,"num_write_kb":61000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213813653504,"internally_reserved":0,"allocated":397840384,"data_stored":397840384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7609,"ondisk_log_size":7609,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213813653504,"internally_reserved":0,"allocated":397840384,"data_stored":397840384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53427671040,"internally_reserved":0,"allocated":125202432,"data_stored":125202432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":120012,"kb_used_data":120012,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52177716,"statfs":{"total":53552873472,"available":53429981184,"internally_reserved":0,"allocated":122892288,"data_stored":122892288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":74356,"kb_used_data":74356,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52223372,"statfs":{"total":53552873472,"available":53476732928,"internally_reserved":0,"allocated":76140544,"data_stored":76140544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":100392,"kb_used_data":100392,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52197336,"statfs":{"total":53552873472,"available":53450072064,"internally_reserved":0,"allocated":102801408,"data_stored":102801408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":86752,"kb_used_data":86752,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52210976,"statfs":{"total":53552873472,"available":53464039424,"internally_reserved":0,"allocated":88834048,"data_stored":88834048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53427671040,"internally_reserved":0,"allocated":125202432,"data_stored":125202432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53462016000,"internally_reserved":0,"allocated":90857472,"data_stored":90857472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53448777728,"internally_reserved":0,"allocated":104095744,"data_stored":104095744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53475188736,"internally_reserved":0,"allocated":77684736,"data_stored":77684736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53427671040,"internally_reserved":0,"allocated":125202432,"data_stored":125202432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53462016000,"internally_reserved":0,"allocated":90857472,"data_stored":90857472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53448777728,"internally_reserved":0,"allocated":104095744,"data_stored":104095744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53475188736,"internally_reserved":0,"allocated":77684736,"data_stored":77684736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53427671040,"internally_reserved":0,"allocated":125202432,"data_stored":125202432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:20.196 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 8095 8079 1.16873 1.14844 0.0345172 0.106735 2024-09-22T03:31:20.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:20.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:20.574+0000 7f7454d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:20.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:20.582+0000 7f7454d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:21.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 8249 8233 1.16935 1.20312 0.123753 0.106775 2024-09-22T03:31:22.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 8394 8378 1.1687 1.13281 0.0623832 0.106837 2024-09-22T03:31:22.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:22.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:22.926+0000 7fda81517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:22.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:22.932+0000 7fda81517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:23.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 8541 8525 1.16834 1.14844 0.167008 0.106835 2024-09-22T03:31:24.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 8695 8679 1.16894 1.20312 0.0788607 0.106854 2024-09-22T03:31:24.371 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:24.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:24.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:24.507+0000 7f6a75317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:24.517+0000 7f6a75317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:31:24.782 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:31:24.797 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-22T03:31:24.027701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69050931,"num_objects":1049,"num_object_clones":0,"num_object_copies":3147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8496,"num_write_kb":68752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8441,"ondisk_log_size":8441,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":418252,"kb_used_data":418252,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208772660,"statfs":{"total":214211493888,"available":213783203840,"internally_reserved":0,"allocated":428290048,"data_stored":428290048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12255232,"num_objects":186,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1496,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002320"},"pg_stats":[{"pgid":"7.f","version":"76'511","reported_seq":548,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.240345+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:31:21.240345+0000","last_peered":"2024-09-22T03:31:21.240345+0000","last_clean":"2024-09-22T03:31:21.240345+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:31:21.240345+0000","last_undegraded":"2024-09-22T03:31:21.240345+0000","last_fullsized":"2024-09-22T03:31:21.240345+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'420","reported_seq":457,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.962751+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:31:21.962751+0000","last_peered":"2024-09-22T03:31:21.962751+0000","last_clean":"2024-09-22T03:31:21.962751+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:31:21.962751+0000","last_undegraded":"2024-09-22T03:31:21.962751+0000","last_fullsized":"2024-09-22T03:31:21.962751+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3448832,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":421,"num_write_kb":3368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'575","reported_seq":612,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.534217+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:31:21.534217+0000","last_peered":"2024-09-22T03:31:21.534217+0000","last_clean":"2024-09-22T03:31:21.534217+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:31:21.534217+0000","last_undegraded":"2024-09-22T03:31:21.534217+0000","last_fullsized":"2024-09-22T03:31:21.534217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'447","reported_seq":484,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:20.675235+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:31:20.675235+0000","last_peered":"2024-09-22T03:31:20.675235+0000","last_clean":"2024-09-22T03:31:20.675235+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:31:20.675235+0000","last_undegraded":"2024-09-22T03:31:20.675235+0000","last_fullsized":"2024-09-22T03:31:20.675235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'527","reported_seq":564,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:19.153563+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:31:19.153563+0000","last_peered":"2024-09-22T03:31:19.153563+0000","last_clean":"2024-09-22T03:31:19.153563+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:31:19.153563+0000","last_undegraded":"2024-09-22T03:31:19.153563+0000","last_fullsized":"2024-09-22T03:31:19.153563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'655","reported_seq":692,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.862945+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:31:21.862945+0000","last_peered":"2024-09-22T03:31:21.862945+0000","last_clean":"2024-09-22T03:31:21.862945+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:31:21.862945+0000","last_undegraded":"2024-09-22T03:31:21.862945+0000","last_fullsized":"2024-09-22T03:31:21.862945+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'559","reported_seq":596,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:18.795579+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:31:18.795579+0000","last_peered":"2024-09-22T03:31:18.795579+0000","last_clean":"2024-09-22T03:31:18.795579+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:31:18.795579+0000","last_undegraded":"2024-09-22T03:31:18.795579+0000","last_fullsized":"2024-09-22T03:31:18.795579+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'439","reported_seq":476,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.746459+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:31:21.746459+0000","last_peered":"2024-09-22T03:31:21.746459+0000","last_clean":"2024-09-22T03:31:21.746459+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:31:21.746459+0000","last_undegraded":"2024-09-22T03:31:21.746459+0000","last_fullsized":"2024-09-22T03:31:21.746459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'455","reported_seq":492,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.183124+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:31:21.183124+0000","last_peered":"2024-09-22T03:31:21.183124+0000","last_clean":"2024-09-22T03:31:21.183124+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:31:21.183124+0000","last_undegraded":"2024-09-22T03:31:21.183124+0000","last_fullsized":"2024-09-22T03:31:21.183124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'575","reported_seq":612,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.376436+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:31:21.376436+0000","last_peered":"2024-09-22T03:31:21.376436+0000","last_clean":"2024-09-22T03:31:21.376436+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:31:21.376436+0000","last_undegraded":"2024-09-22T03:31:21.376436+0000","last_fullsized":"2024-09-22T03:31:21.376436+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'455","reported_seq":492,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.289870+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:31:21.289870+0000","last_peered":"2024-09-22T03:31:21.289870+0000","last_clean":"2024-09-22T03:31:21.289870+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:31:21.289870+0000","last_undegraded":"2024-09-22T03:31:21.289870+0000","last_fullsized":"2024-09-22T03:31:21.289870+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'487","reported_seq":524,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:20.924060+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:31:20.924060+0000","last_peered":"2024-09-22T03:31:20.924060+0000","last_clean":"2024-09-22T03:31:20.924060+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:31:20.924060+0000","last_undegraded":"2024-09-22T03:31:20.924060+0000","last_fullsized":"2024-09-22T03:31:20.924060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'551","reported_seq":588,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.137165+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:31:21.137165+0000","last_peered":"2024-09-22T03:31:21.137165+0000","last_clean":"2024-09-22T03:31:21.137165+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:31:21.137165+0000","last_undegraded":"2024-09-22T03:31:21.137165+0000","last_fullsized":"2024-09-22T03:31:21.137165+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'447","reported_seq":484,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:20.527198+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:31:20.527198+0000","last_peered":"2024-09-22T03:31:20.527198+0000","last_clean":"2024-09-22T03:31:20.527198+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:31:20.527198+0000","last_undegraded":"2024-09-22T03:31:20.527198+0000","last_fullsized":"2024-09-22T03:31:20.527198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'671","reported_seq":708,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.598558+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:31:21.598558+0000","last_peered":"2024-09-22T03:31:21.598558+0000","last_clean":"2024-09-22T03:31:21.598558+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:31:21.598558+0000","last_undegraded":"2024-09-22T03:31:21.598558+0000","last_fullsized":"2024-09-22T03:31:21.598558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'567","reported_seq":604,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:21.922146+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:31:21.922146+0000","last_peered":"2024-09-22T03:31:21.922146+0000","last_clean":"2024-09-22T03:31:21.922146+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:31:21.922146+0000","last_undegraded":"2024-09-22T03:31:21.922146+0000","last_fullsized":"2024-09-22T03:31:21.922146+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":68460544,"num_objects":1045,"num_object_clones":0,"num_object_copies":3135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8357,"num_write_kb":66856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213776912384,"internally_reserved":0,"allocated":434581504,"data_stored":434581504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8341,"ondisk_log_size":8341,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213776928768,"internally_reserved":0,"allocated":434565120,"data_stored":434565120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53415829504,"internally_reserved":0,"allocated":137043968,"data_stored":137043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":131680,"kb_used_data":131680,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166048,"statfs":{"total":53552873472,"available":53418033152,"internally_reserved":0,"allocated":134840320,"data_stored":134840320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":81292,"kb_used_data":81292,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52216436,"statfs":{"total":53552873472,"available":53469630464,"internally_reserved":0,"allocated":83243008,"data_stored":83243008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":109188,"kb_used_data":109188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52188540,"statfs":{"total":53552873472,"available":53441064960,"internally_reserved":0,"allocated":111808512,"data_stored":111808512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":96092,"kb_used_data":96092,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52201636,"statfs":{"total":53552873472,"available":53454475264,"internally_reserved":0,"allocated":98398208,"data_stored":98398208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53415829504,"internally_reserved":0,"allocated":137043968,"data_stored":137043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53452828672,"internally_reserved":0,"allocated":100044800,"data_stored":100044800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53439627264,"internally_reserved":0,"allocated":113246208,"data_stored":113246208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53468643328,"internally_reserved":0,"allocated":84230144,"data_stored":84230144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53415829504,"internally_reserved":0,"allocated":137043968,"data_stored":137043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53452820480,"internally_reserved":0,"allocated":100052992,"data_stored":100052992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53439627264,"internally_reserved":0,"allocated":113246208,"data_stored":113246208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53468635136,"internally_reserved":0,"allocated":84238336,"data_stored":84238336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53415829504,"internally_reserved":0,"allocated":137043968,"data_stored":137043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:25.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:25.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 8837 8821 1.16793 1.10938 0.105198 0.106931 2024-09-22T03:31:25.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:25.227+0000 7fba610d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:25.232+0000 7fba610d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:26.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:31:26.196952+0000 min lat: 0.0135712 max lat: 0.431193 avg lat: 0.106879 2024-09-22T03:31:26.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:31:26.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 8991 8975 1.16851 1.20312 0.158505 0.106879 2024-09-22T03:31:27.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 9153 9137 1.1701 1.26562 0.0775681 0.106735 2024-09-22T03:31:27.381 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:27.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:27.506+0000 7f614c70e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:27.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:27.512+0000 7f614c70e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:28.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 9285 9269 1.16786 1.03125 0.0253252 0.106845 2024-09-22T03:31:29.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 9424 9408 1.16656 1.08594 0.183475 0.106998 2024-09-22T03:31:29.639 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:29.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:29.757+0000 7fddd8d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:29.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:29.762+0000 7fddd8d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:29.800 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:29.800 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:29.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:29.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:29.937+0000 7f3aa8757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:29.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:29.945+0000 7f3aa8757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:30.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 16 9556 9540 1.16444 1.03125 0.0776334 0.107244 2024-09-22T03:31:30.215 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:31:30.216 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:31:30.233 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-22T03:31:30.028814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75235891,"num_objects":1143,"num_object_clones":0,"num_object_copies":3429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":9251,"num_write_kb":74792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9196,"ondisk_log_size":9196,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":455820,"kb_used_data":455820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208735092,"statfs":{"total":214211493888,"available":213744734208,"internally_reserved":0,"allocated":466759680,"data_stored":466759680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12181504,"num_objects":185,"num_object_clones":0,"num_object_copies":555,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002264"},"pg_stats":[{"pgid":"7.f","version":"76'543","reported_seq":580,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:25.772105+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:31:25.772105+0000","last_peered":"2024-09-22T03:31:25.772105+0000","last_clean":"2024-09-22T03:31:25.772105+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:31:25.772105+0000","last_undegraded":"2024-09-22T03:31:25.772105+0000","last_fullsized":"2024-09-22T03:31:25.772105+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'463","reported_seq":500,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.755476+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:31:26.755476+0000","last_peered":"2024-09-22T03:31:26.755476+0000","last_clean":"2024-09-22T03:31:26.755476+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:31:26.755476+0000","last_undegraded":"2024-09-22T03:31:26.755476+0000","last_fullsized":"2024-09-22T03:31:26.755476+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'599","reported_seq":636,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:25.356456+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:31:25.356456+0000","last_peered":"2024-09-22T03:31:25.356456+0000","last_clean":"2024-09-22T03:31:25.356456+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:31:25.356456+0000","last_undegraded":"2024-09-22T03:31:25.356456+0000","last_fullsized":"2024-09-22T03:31:25.356456+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'479","reported_seq":516,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.588970+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:31:26.588970+0000","last_peered":"2024-09-22T03:31:26.588970+0000","last_clean":"2024-09-22T03:31:26.588970+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:31:26.588970+0000","last_undegraded":"2024-09-22T03:31:26.588970+0000","last_fullsized":"2024-09-22T03:31:26.588970+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'615","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.889386+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:31:26.889386+0000","last_peered":"2024-09-22T03:31:26.889386+0000","last_clean":"2024-09-22T03:31:26.889386+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:31:26.889386+0000","last_undegraded":"2024-09-22T03:31:26.889386+0000","last_fullsized":"2024-09-22T03:31:26.889386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'671","reported_seq":708,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.117119+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:31:26.117119+0000","last_peered":"2024-09-22T03:31:26.117119+0000","last_clean":"2024-09-22T03:31:26.117119+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:31:26.117119+0000","last_undegraded":"2024-09-22T03:31:26.117119+0000","last_fullsized":"2024-09-22T03:31:26.117119+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'607","reported_seq":644,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.936926+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:31:26.936926+0000","last_peered":"2024-09-22T03:31:26.936926+0000","last_clean":"2024-09-22T03:31:26.936926+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:31:26.936926+0000","last_undegraded":"2024-09-22T03:31:26.936926+0000","last_fullsized":"2024-09-22T03:31:26.936926+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'487","reported_seq":524,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:25.041929+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:31:25.041929+0000","last_peered":"2024-09-22T03:31:25.041929+0000","last_clean":"2024-09-22T03:31:25.041929+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:31:25.041929+0000","last_undegraded":"2024-09-22T03:31:25.041929+0000","last_fullsized":"2024-09-22T03:31:25.041929+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'527","reported_seq":564,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.797154+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:31:26.797154+0000","last_peered":"2024-09-22T03:31:26.797154+0000","last_clean":"2024-09-22T03:31:26.797154+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:31:26.797154+0000","last_undegraded":"2024-09-22T03:31:26.797154+0000","last_fullsized":"2024-09-22T03:31:26.797154+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'623","reported_seq":660,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.008255+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:31:26.008255+0000","last_peered":"2024-09-22T03:31:26.008255+0000","last_clean":"2024-09-22T03:31:26.008255+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:31:26.008255+0000","last_undegraded":"2024-09-22T03:31:26.008255+0000","last_fullsized":"2024-09-22T03:31:26.008255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'511","reported_seq":548,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.970629+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:31:26.970629+0000","last_peered":"2024-09-22T03:31:26.970629+0000","last_clean":"2024-09-22T03:31:26.970629+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:31:26.970629+0000","last_undegraded":"2024-09-22T03:31:26.970629+0000","last_fullsized":"2024-09-22T03:31:26.970629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'511","reported_seq":548,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.695894+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:31:26.695894+0000","last_peered":"2024-09-22T03:31:26.695894+0000","last_clean":"2024-09-22T03:31:26.695894+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:31:26.695894+0000","last_undegraded":"2024-09-22T03:31:26.695894+0000","last_fullsized":"2024-09-22T03:31:26.695894+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'607","reported_seq":644,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.593821+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:31:26.593821+0000","last_peered":"2024-09-22T03:31:26.593821+0000","last_clean":"2024-09-22T03:31:26.593821+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:31:26.593821+0000","last_undegraded":"2024-09-22T03:31:26.593821+0000","last_fullsized":"2024-09-22T03:31:26.593821+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'527","reported_seq":564,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:25.877650+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:31:25.877650+0000","last_peered":"2024-09-22T03:31:25.877650+0000","last_clean":"2024-09-22T03:31:25.877650+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:31:25.877650+0000","last_undegraded":"2024-09-22T03:31:25.877650+0000","last_fullsized":"2024-09-22T03:31:25.877650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'703","reported_seq":740,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.073486+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:31:26.073486+0000","last_peered":"2024-09-22T03:31:26.073486+0000","last_clean":"2024-09-22T03:31:26.073486+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:31:26.073486+0000","last_undegraded":"2024-09-22T03:31:26.073486+0000","last_fullsized":"2024-09-22T03:31:26.073486+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'623","reported_seq":660,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:26.320629+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:31:26.320629+0000","last_peered":"2024-09-22T03:31:26.320629+0000","last_clean":"2024-09-22T03:31:26.320629+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:31:26.320629+0000","last_undegraded":"2024-09-22T03:31:26.320629+0000","last_fullsized":"2024-09-22T03:31:26.320629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":74645504,"num_objects":1139,"num_object_clones":0,"num_object_copies":3417,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9112,"num_write_kb":72896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213737197568,"internally_reserved":0,"allocated":474296320,"data_stored":474296320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9096,"ondisk_log_size":9096,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213737197568,"internally_reserved":0,"allocated":474296320,"data_stored":474296320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53403410432,"internally_reserved":0,"allocated":149463040,"data_stored":149463040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":143620,"kb_used_data":143620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52154108,"statfs":{"total":53552873472,"available":53405806592,"internally_reserved":0,"allocated":147066880,"data_stored":147066880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":87940,"kb_used_data":87940,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52209788,"statfs":{"total":53552873472,"available":53462822912,"internally_reserved":0,"allocated":90050560,"data_stored":90050560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":119688,"kb_used_data":119688,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52178040,"statfs":{"total":53552873472,"available":53430312960,"internally_reserved":0,"allocated":122560512,"data_stored":122560512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":104572,"kb_used_data":104572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52193156,"statfs":{"total":53552873472,"available":53445791744,"internally_reserved":0,"allocated":107081728,"data_stored":107081728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53403410432,"internally_reserved":0,"allocated":149463040,"data_stored":149463040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53443674112,"internally_reserved":0,"allocated":109199360,"data_stored":109199360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53428121600,"internally_reserved":0,"allocated":124751872,"data_stored":124751872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53462065152,"internally_reserved":0,"allocated":90808320,"data_stored":90808320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53403336704,"internally_reserved":0,"allocated":149536768,"data_stored":149536768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53443674112,"internally_reserved":0,"allocated":109199360,"data_stored":109199360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53428121600,"internally_reserved":0,"allocated":124751872,"data_stored":124751872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53462065152,"internally_reserved":0,"allocated":90808320,"data_stored":90808320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53403336704,"internally_reserved":0,"allocated":149536768,"data_stored":149536768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:31.197 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 9668 9652 1.15999 0.875 0.165292 0.107652 2024-09-22T03:31:32.027 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:32.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:32.140+0000 7fe40e863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:32.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:32.146+0000 7fe40e863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:32.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 9818 9802 1.16017 1.17188 0.0511476 0.107644 2024-09-22T03:31:33.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 9946 9930 1.15777 1 0.142572 0.107851 2024-09-22T03:31:34.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 10079 10063 1.15603 1.03906 0.12168 0.107955 2024-09-22T03:31:34.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:34.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:34.451+0000 7fdf536c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:34.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:34.458+0000 7fdf536c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 10204 10188 1.15342 0.976562 0.151358 0.108226 2024-09-22T03:31:35.235 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:35.236 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:35.236 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:31:35.236 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:31:35.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.388+0000 7f7a3b6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.398+0000 7f7a3b6b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.437 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:35.437 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:35.437 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:35.437 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:35.438 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:35.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:35.441 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:35.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:31:35.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.574+0000 7f4059197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.583+0000 7f4059197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.598 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:35.599 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:35.602 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:35.603 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:35.603 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:31:35.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.746+0000 7fc3ce26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.755+0000 7fc3ce26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.770 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:35.770 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:35.770 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:35.770 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:35.770 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:35.771 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:35.771 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:35.771 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:35.771 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:35.771 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:35.771 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:35.771 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:35.773 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:35.773 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:35.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:31:35.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.912+0000 7f5852d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:35.924+0000 7f5852d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:35.974 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:35.975 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:35.978 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:35.978 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:36.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 10341 10325 1.15223 1.07031 0.238997 0.108314 2024-09-22T03:31:36.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:36.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:36.831+0000 7f3ba6117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:36.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:36.837+0000 7f3ba6117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:37.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 10478 10462 1.15108 1.07031 0.126781 0.108483 2024-09-22T03:31:38.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 10601 10585 1.14844 0.960938 0.224882 0.108661 2024-09-22T03:31:39.000 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:39.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:39.131+0000 7f2840e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:39.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:39.139+0000 7f2840e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:39.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 10728 10712 1.1463 0.992188 0.0888248 0.108921 2024-09-22T03:31:40.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 10865 10849 1.14527 1.07031 0.170206 0.109024 2024-09-22T03:31:40.979 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:40.979 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:40.979 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:31:40.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:31:41.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.108+0000 7f05a0b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.121+0000 7f05a0b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:41.185 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:41.186 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:41.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:41.188 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:41.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:31:41.198 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 11014 10998 1.14552 1.16406 0.11213 0.109028 2024-09-22T03:31:41.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.304+0000 7efebd997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.314+0000 7efebd997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.322 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:41.343 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:41.343 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:41.343 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:41.343 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:41.343 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:41.343 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:41.343 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:41.344 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:41.344 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:41.344 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:41.344 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:41.344 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:41.347 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:41.348 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:41.348 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:31:41.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.435+0000 7fb508597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.440+0000 7fb508597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.515+0000 7f6566f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.528+0000 7f6566f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.547 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:41.547 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:41.547 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:41.547 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:41.548 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:41.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:41.551 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:41.551 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:31:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.674+0000 7fabf8917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:41.682+0000 7fabf8917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:41.726 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:41.728 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:41.729 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:42.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 11164 11148 1.14586 1.17188 0.136929 0.109002 2024-09-22T03:31:43.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 11299 11283 1.14468 1.05469 0.180912 0.109079 2024-09-22T03:31:43.591 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:43.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:43.731+0000 7f380ceb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:43.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:43.737+0000 7f380ceb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:44.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 16 11426 11410 1.14272 0.992188 0.138259 0.109239 2024-09-22T03:31:45.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 11572 11556 1.14269 1.14062 0.0433433 0.109261 2024-09-22T03:31:45.907 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:46.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:46.041+0000 7fc849397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:46.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:46.047+0000 7fc849397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:46.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:31:46.198773+0000 min lat: 0.0135712 max lat: 0.431193 avg lat: 0.109441 2024-09-22T03:31:46.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:31:46.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 11701 11685 1.14101 1.00781 0.171636 0.109441 2024-09-22T03:31:46.729 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:46.729 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:46.730 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:46.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:46.845+0000 7f4be2c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:46.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:46.852+0000 7f4be2c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:47.153 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:31:47.154 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:31:47.173 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-22T03:31:46.031876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91734579,"num_objects":1395,"num_object_clones":0,"num_object_copies":4185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11265,"num_write_kb":90904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11210,"ondisk_log_size":11210,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":557020,"kb_used_data":557020,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208633892,"statfs":{"total":214211493888,"available":213641105408,"internally_reserved":0,"allocated":570388480,"data_stored":570388480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11051008,"num_objects":168,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1349,"num_write_kb":10792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002383"},"pg_stats":[{"pgid":"7.f","version":"76'671","reported_seq":708,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:40.992500+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:31:40.992500+0000","last_peered":"2024-09-22T03:31:40.992500+0000","last_clean":"2024-09-22T03:31:40.992500+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:31:40.992500+0000","last_undegraded":"2024-09-22T03:31:40.992500+0000","last_fullsized":"2024-09-22T03:31:40.992500+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'599","reported_seq":636,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:39.461026+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:31:39.461026+0000","last_peered":"2024-09-22T03:31:39.461026+0000","last_clean":"2024-09-22T03:31:39.461026+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:31:39.461026+0000","last_undegraded":"2024-09-22T03:31:39.461026+0000","last_fullsized":"2024-09-22T03:31:39.461026+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'679","reported_seq":716,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.296285+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:31:41.296285+0000","last_peered":"2024-09-22T03:31:41.296285+0000","last_clean":"2024-09-22T03:31:41.296285+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:31:41.296285+0000","last_undegraded":"2024-09-22T03:31:41.296285+0000","last_fullsized":"2024-09-22T03:31:41.296285+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'583","reported_seq":620,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.690763+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:31:41.690763+0000","last_peered":"2024-09-22T03:31:41.690763+0000","last_clean":"2024-09-22T03:31:41.690763+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:31:41.690763+0000","last_undegraded":"2024-09-22T03:31:41.690763+0000","last_fullsized":"2024-09-22T03:31:41.690763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'727","reported_seq":764,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.062112+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:31:41.062112+0000","last_peered":"2024-09-22T03:31:41.062112+0000","last_clean":"2024-09-22T03:31:41.062112+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:31:41.062112+0000","last_undegraded":"2024-09-22T03:31:41.062112+0000","last_fullsized":"2024-09-22T03:31:41.062112+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'799","reported_seq":836,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.386439+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:31:41.386439+0000","last_peered":"2024-09-22T03:31:41.386439+0000","last_clean":"2024-09-22T03:31:41.386439+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:31:41.386439+0000","last_undegraded":"2024-09-22T03:31:41.386439+0000","last_fullsized":"2024-09-22T03:31:41.386439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'743","reported_seq":780,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:40.591602+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:31:40.591602+0000","last_peered":"2024-09-22T03:31:40.591602+0000","last_clean":"2024-09-22T03:31:40.591602+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:31:40.591602+0000","last_undegraded":"2024-09-22T03:31:40.591602+0000","last_fullsized":"2024-09-22T03:31:40.591602+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'639","reported_seq":676,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.877277+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:31:41.877277+0000","last_peered":"2024-09-22T03:31:41.877277+0000","last_clean":"2024-09-22T03:31:41.877277+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:31:41.877277+0000","last_undegraded":"2024-09-22T03:31:41.877277+0000","last_fullsized":"2024-09-22T03:31:41.877277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'679","reported_seq":716,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.796814+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:31:41.796814+0000","last_peered":"2024-09-22T03:31:41.796814+0000","last_clean":"2024-09-22T03:31:41.796814+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:31:41.796814+0000","last_undegraded":"2024-09-22T03:31:41.796814+0000","last_fullsized":"2024-09-22T03:31:41.796814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'751","reported_seq":788,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:39.952808+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:31:39.952808+0000","last_peered":"2024-09-22T03:31:39.952808+0000","last_clean":"2024-09-22T03:31:39.952808+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:31:39.952808+0000","last_undegraded":"2024-09-22T03:31:39.952808+0000","last_fullsized":"2024-09-22T03:31:39.952808+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'621","reported_seq":658,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.981639+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:31:41.981639+0000","last_peered":"2024-09-22T03:31:41.981639+0000","last_clean":"2024-09-22T03:31:41.981639+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:31:41.981639+0000","last_undegraded":"2024-09-22T03:31:41.981639+0000","last_fullsized":"2024-09-22T03:31:41.981639+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095424,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":622,"num_write_kb":4976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'623","reported_seq":660,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:39.576595+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:31:39.576595+0000","last_peered":"2024-09-22T03:31:39.576595+0000","last_clean":"2024-09-22T03:31:39.576595+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:31:39.576595+0000","last_undegraded":"2024-09-22T03:31:39.576595+0000","last_fullsized":"2024-09-22T03:31:39.576595+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'735","reported_seq":772,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.545742+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:31:41.545742+0000","last_peered":"2024-09-22T03:31:41.545742+0000","last_clean":"2024-09-22T03:31:41.545742+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:31:41.545742+0000","last_undegraded":"2024-09-22T03:31:41.545742+0000","last_fullsized":"2024-09-22T03:31:41.545742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'607","reported_seq":644,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.918113+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:31:41.918113+0000","last_peered":"2024-09-22T03:31:41.918113+0000","last_clean":"2024-09-22T03:31:41.918113+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:31:41.918113+0000","last_undegraded":"2024-09-22T03:31:41.918113+0000","last_fullsized":"2024-09-22T03:31:41.918113+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'871","reported_seq":908,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.844907+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:31:41.844907+0000","last_peered":"2024-09-22T03:31:41.844907+0000","last_clean":"2024-09-22T03:31:41.844907+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:31:41.844907+0000","last_undegraded":"2024-09-22T03:31:41.844907+0000","last_fullsized":"2024-09-22T03:31:41.844907+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'783","reported_seq":820,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:41.774853+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:31:41.774853+0000","last_peered":"2024-09-22T03:31:41.774853+0000","last_clean":"2024-09-22T03:31:41.774853+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:31:41.774853+0000","last_undegraded":"2024-09-22T03:31:41.774853+0000","last_fullsized":"2024-09-22T03:31:41.774853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":91144192,"num_objects":1391,"num_object_clones":0,"num_object_copies":4173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11126,"num_write_kb":89008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213634486272,"internally_reserved":0,"allocated":577007616,"data_stored":577007616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11110,"ondisk_log_size":11110,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213634502656,"internally_reserved":0,"allocated":576991232,"data_stored":576991232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53371531264,"internally_reserved":0,"allocated":181342208,"data_stored":181342208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":174852,"kb_used_data":174852,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52122876,"statfs":{"total":53552873472,"available":53373825024,"internally_reserved":0,"allocated":179048448,"data_stored":179048448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":105228,"kb_used_data":105228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52192500,"statfs":{"total":53552873472,"available":53445120000,"internally_reserved":0,"allocated":107753472,"data_stored":107753472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":147132,"kb_used_data":147132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52150596,"statfs":{"total":53552873472,"available":53402210304,"internally_reserved":0,"allocated":150663168,"data_stored":150663168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":129808,"kb_used_data":129808,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52167920,"statfs":{"total":53552873472,"available":53419950080,"internally_reserved":0,"allocated":132923392,"data_stored":132923392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53371531264,"internally_reserved":0,"allocated":181342208,"data_stored":181342208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53418139648,"internally_reserved":0,"allocated":134733824,"data_stored":134733824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53400940544,"internally_reserved":0,"allocated":151932928,"data_stored":151932928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53443891200,"internally_reserved":0,"allocated":108982272,"data_stored":108982272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53371531264,"internally_reserved":0,"allocated":181342208,"data_stored":181342208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53418139648,"internally_reserved":0,"allocated":134733824,"data_stored":134733824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53400940544,"internally_reserved":0,"allocated":151932928,"data_stored":151932928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53443883008,"internally_reserved":0,"allocated":108990464,"data_stored":108990464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53371523072,"internally_reserved":0,"allocated":181350400,"data_stored":181350400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:47.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 11838 11822 1.14013 1.07031 0.108774 0.109517 2024-09-22T03:31:48.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 12001 11985 1.14176 1.27344 0.0851086 0.1094 2024-09-22T03:31:48.211 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:48.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:48.334+0000 7fe8a7ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:48.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:48.340+0000 7fe8a7ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:49.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 12118 12102 1.13901 0.914062 0.215264 0.109606 2024-09-22T03:31:50.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 12264 12248 1.13903 1.14062 0.140115 0.109606 2024-09-22T03:31:50.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:50.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:50.645+0000 7fd2eda18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:50.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:50.653+0000 7fd2eda18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:51.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 12401 12385 1.13822 1.07031 0.292219 0.109717 2024-09-22T03:31:52.175 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:52.175 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:52.175 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:31:52.176 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:31:52.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 12525 12509 1.13625 0.96875 0.185205 0.109891 2024-09-22T03:31:52.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.298+0000 7fc3b4997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.308+0000 7fc3b4997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.353 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:52.353 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:52.356 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:52.357 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:52.357 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:31:52.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.539+0000 7fc1b7e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.548+0000 7fc1b7e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.577 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:52.577 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:52.578 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:52.581 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:52.581 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:52.581 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:31:52.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.690+0000 7f45eef57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.699+0000 7f45eef57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:52.715 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:52.718 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:52.718 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:52.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:31:52.837 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:52.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.849+0000 7f6698f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.864+0000 7f6698f17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:31:52.886 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:31:52.887 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:31:52.887 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:31:52.887 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:31:52.890 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:31:52.891 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:31:52.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.962+0000 7f4cba557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:52.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:52.969+0000 7f4cba557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:53.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 12659 12643 1.13522 1.04688 0.04034 0.109992 2024-09-22T03:31:54.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 12814 12798 1.13608 1.21094 0.0849958 0.109961 2024-09-22T03:31:55.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:31:55.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 12958 12942 1.13596 1.125 0.0363852 0.109927 2024-09-22T03:31:55.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:55.256+0000 7fe1a3997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:55.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:55.262+0000 7fe1a3997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:56.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 16 13079 13063 1.13384 0.945312 0.0491811 0.110083 2024-09-22T03:31:57.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.0966 2024-09-22T03:31:57.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 13079 2024-09-22T03:31:57.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:31:57.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.13411 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.10186 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.35938 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 0.875 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 145 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 13.038 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 174 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 112 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.110167 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.0594511 2024-09-22T03:31:57.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.431193 2024-09-22T03:31:57.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0135712 2024-09-22T03:31:57.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:31:57.587 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:31:57.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:57.715+0000 7f4d0ff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:57.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:57.720+0000 7f4d0ff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:57.892 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:31:57.892 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:31:57.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:31:58.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:58.044+0000 7f166a557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:58.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:31:58.055+0000 7f166a557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:31:58.318 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:31:58.319 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:31:58.339 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-22T03:31:58.034282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107733555,"num_objects":1639,"num_object_clones":0,"num_object_copies":4917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13218,"num_write_kb":106528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13163,"ondisk_log_size":13163,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":660460,"kb_used_data":660460,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208530452,"statfs":{"total":214211493888,"available":213535182848,"internally_reserved":0,"allocated":676311040,"data_stored":676311040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15998976,"num_objects":244,"num_object_clones":0,"num_object_copies":732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1953,"num_write_kb":15624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002405"},"pg_stats":[{"pgid":"7.f","version":"76'799","reported_seq":836,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:55.506756+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:31:55.506756+0000","last_peered":"2024-09-22T03:31:55.506756+0000","last_clean":"2024-09-22T03:31:55.506756+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:31:55.506756+0000","last_undegraded":"2024-09-22T03:31:55.506756+0000","last_fullsized":"2024-09-22T03:31:55.506756+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'671","reported_seq":708,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:52.407418+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:31:52.407418+0000","last_peered":"2024-09-22T03:31:52.407418+0000","last_clean":"2024-09-22T03:31:52.407418+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:31:52.407418+0000","last_undegraded":"2024-09-22T03:31:52.407418+0000","last_fullsized":"2024-09-22T03:31:52.407418+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'807","reported_seq":844,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:54.627572+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:31:54.627572+0000","last_peered":"2024-09-22T03:31:54.627572+0000","last_clean":"2024-09-22T03:31:54.627572+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:31:54.627572+0000","last_undegraded":"2024-09-22T03:31:54.627572+0000","last_fullsized":"2024-09-22T03:31:54.627572+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'719","reported_seq":756,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:55.393855+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:31:55.393855+0000","last_peered":"2024-09-22T03:31:55.393855+0000","last_clean":"2024-09-22T03:31:55.393855+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:31:55.393855+0000","last_undegraded":"2024-09-22T03:31:55.393855+0000","last_fullsized":"2024-09-22T03:31:55.393855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'854","reported_seq":891,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:56.191649+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:31:56.191649+0000","last_peered":"2024-09-22T03:31:56.191649+0000","last_clean":"2024-09-22T03:31:56.191649+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:31:56.191649+0000","last_undegraded":"2024-09-22T03:31:56.191649+0000","last_fullsized":"2024-09-22T03:31:56.191649+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7004160,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":855,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'959","reported_seq":996,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:55.901592+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:31:55.901592+0000","last_peered":"2024-09-22T03:31:55.901592+0000","last_clean":"2024-09-22T03:31:55.901592+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:31:55.901592+0000","last_undegraded":"2024-09-22T03:31:55.901592+0000","last_fullsized":"2024-09-22T03:31:55.901592+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":959,"log_dups_size":0,"ondisk_log_size":959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7864320,"num_objects":120,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":960,"num_write_kb":7680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'871","reported_seq":908,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:55.304971+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:31:55.304971+0000","last_peered":"2024-09-22T03:31:55.304971+0000","last_clean":"2024-09-22T03:31:55.304971+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:31:55.304971+0000","last_undegraded":"2024-09-22T03:31:55.304971+0000","last_fullsized":"2024-09-22T03:31:55.304971+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'767","reported_seq":804,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:54.747746+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:31:54.747746+0000","last_peered":"2024-09-22T03:31:54.747746+0000","last_clean":"2024-09-22T03:31:54.747746+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:31:54.747746+0000","last_undegraded":"2024-09-22T03:31:54.747746+0000","last_fullsized":"2024-09-22T03:31:54.747746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'823","reported_seq":860,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:56.105919+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:31:56.105919+0000","last_peered":"2024-09-22T03:31:56.105919+0000","last_clean":"2024-09-22T03:31:56.105919+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:31:56.105919+0000","last_undegraded":"2024-09-22T03:31:56.105919+0000","last_fullsized":"2024-09-22T03:31:56.105919+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'847","reported_seq":884,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:53.181588+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:31:53.181588+0000","last_peered":"2024-09-22T03:31:53.181588+0000","last_clean":"2024-09-22T03:31:53.181588+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:31:53.181588+0000","last_undegraded":"2024-09-22T03:31:53.181588+0000","last_fullsized":"2024-09-22T03:31:53.181588+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'711","reported_seq":748,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:53.777102+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:31:53.777102+0000","last_peered":"2024-09-22T03:31:53.777102+0000","last_clean":"2024-09-22T03:31:53.777102+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:31:53.777102+0000","last_undegraded":"2024-09-22T03:31:53.777102+0000","last_fullsized":"2024-09-22T03:31:53.777102+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'695","reported_seq":732,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:54.864058+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:31:54.864058+0000","last_peered":"2024-09-22T03:31:54.864058+0000","last_clean":"2024-09-22T03:31:54.864058+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:31:54.864058+0000","last_undegraded":"2024-09-22T03:31:54.864058+0000","last_fullsized":"2024-09-22T03:31:54.864058+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'871","reported_seq":908,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:54.658206+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:31:54.658206+0000","last_peered":"2024-09-22T03:31:54.658206+0000","last_clean":"2024-09-22T03:31:54.658206+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:31:54.658206+0000","last_undegraded":"2024-09-22T03:31:54.658206+0000","last_fullsized":"2024-09-22T03:31:54.658206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7143424,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":872,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'727","reported_seq":764,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:56.013402+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:31:56.013402+0000","last_peered":"2024-09-22T03:31:56.013402+0000","last_clean":"2024-09-22T03:31:56.013402+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:31:56.013402+0000","last_undegraded":"2024-09-22T03:31:56.013402+0000","last_fullsized":"2024-09-22T03:31:56.013402+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'1007","reported_seq":1044,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:55.066955+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:31:55.066955+0000","last_peered":"2024-09-22T03:31:55.066955+0000","last_clean":"2024-09-22T03:31:55.066955+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:31:55.066955+0000","last_undegraded":"2024-09-22T03:31:55.066955+0000","last_fullsized":"2024-09-22T03:31:55.066955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8257536,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'935","reported_seq":972,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:31:55.964608+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:31:55.964608+0000","last_peered":"2024-09-22T03:31:55.964608+0000","last_clean":"2024-09-22T03:31:55.964608+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:31:55.964608+0000","last_undegraded":"2024-09-22T03:31:55.964608+0000","last_fullsized":"2024-09-22T03:31:55.964608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":936,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":107143168,"num_objects":1635,"num_object_clones":0,"num_object_copies":4905,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":104632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213534552064,"internally_reserved":0,"allocated":676941824,"data_stored":676941824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13063,"ondisk_log_size":13063,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213534552064,"internally_reserved":0,"allocated":676941824,"data_stored":676941824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53339312128,"internally_reserved":0,"allocated":213561344,"data_stored":213561344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":208148,"kb_used_data":208148,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52089580,"statfs":{"total":53552873472,"available":53339729920,"internally_reserved":0,"allocated":213143552,"data_stored":213143552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":124408,"kb_used_data":124408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52173320,"statfs":{"total":53552873472,"available":53425479680,"internally_reserved":0,"allocated":127393792,"data_stored":127393792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":174068,"kb_used_data":174068,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52123660,"statfs":{"total":53552873472,"available":53374627840,"internally_reserved":0,"allocated":178245632,"data_stored":178245632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":153836,"kb_used_data":153836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52143892,"statfs":{"total":53552873472,"available":53395345408,"internally_reserved":0,"allocated":157528064,"data_stored":157528064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53339312128,"internally_reserved":0,"allocated":213561344,"data_stored":213561344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53395288064,"internally_reserved":0,"allocated":157585408,"data_stored":157585408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53374472192,"internally_reserved":0,"allocated":178401280,"data_stored":178401280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53425479680,"internally_reserved":0,"allocated":127393792,"data_stored":127393792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53339312128,"internally_reserved":0,"allocated":213561344,"data_stored":213561344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53395288064,"internally_reserved":0,"allocated":157585408,"data_stored":157585408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53374472192,"internally_reserved":0,"allocated":178401280,"data_stored":178401280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53425479680,"internally_reserved":0,"allocated":127393792,"data_stored":127393792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53339312128,"internally_reserved":0,"allocated":213561344,"data_stored":213561344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:31:59.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:00.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:00.122+0000 7f992de2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:00.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:00.127+0000 7f992de2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:02.417 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:02.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:02.556+0000 7f175486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:02.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:02.563+0000 7f175486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:03.340 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:03.341 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:03.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:32:03.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:03.490+0000 7f55ae2b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:03.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:03.502+0000 7f55ae2b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:03.791 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:32:03.791 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:32:03.813 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-22T03:32:02.035146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":56615499,"num_objects":860,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13999,"num_write_kb":106529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13944,"ondisk_log_size":13944,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":412928,"kb_used_data":412928,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208777984,"statfs":{"total":214211493888,"available":213788655616,"internally_reserved":0,"allocated":422838272,"data_stored":422838272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-40722408,"num_objects":-621,"num_object_clones":0,"num_object_copies":-1863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2050,"num_write_kb":10153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002437"},"pg_stats":[{"pgid":"7.f","version":"76'847","reported_seq":884,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.682814+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:32:01.682814+0000","last_peered":"2024-09-22T03:32:01.682814+0000","last_clean":"2024-09-22T03:32:01.682814+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:32:01.682814+0000","last_undegraded":"2024-09-22T03:32:01.682814+0000","last_fullsized":"2024-09-22T03:32:01.682814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'708","reported_seq":745,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.812919+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:32:01.812919+0000","last_peered":"2024-09-22T03:32:01.812919+0000","last_clean":"2024-09-22T03:32:01.812919+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:32:01.812919+0000","last_undegraded":"2024-09-22T03:32:01.812919+0000","last_fullsized":"2024-09-22T03:32:01.812919+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":709,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'863","reported_seq":901,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.926917+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:32:01.926917+0000","last_peered":"2024-09-22T03:32:01.926917+0000","last_clean":"2024-09-22T03:32:01.926917+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:32:01.926917+0000","last_undegraded":"2024-09-22T03:32:01.926917+0000","last_fullsized":"2024-09-22T03:32:01.926917+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014680,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":864,"num_write_kb":6465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'765","reported_seq":802,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.692504+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:32:01.692504+0000","last_peered":"2024-09-22T03:32:01.692504+0000","last_clean":"2024-09-22T03:32:01.692504+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:32:01.692504+0000","last_undegraded":"2024-09-22T03:32:01.692504+0000","last_fullsized":"2024-09-22T03:32:01.692504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":766,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'904","reported_seq":941,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.856959+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:32:01.856959+0000","last_peered":"2024-09-22T03:32:01.856959+0000","last_clean":"2024-09-22T03:32:01.856959+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:32:01.856959+0000","last_undegraded":"2024-09-22T03:32:01.856959+0000","last_fullsized":"2024-09-22T03:32:01.856959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727360,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":905,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'1021","reported_seq":1058,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.811129+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:32:01.811129+0000","last_peered":"2024-09-22T03:32:01.811129+0000","last_clean":"2024-09-22T03:32:01.811129+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:32:01.811129+0000","last_undegraded":"2024-09-22T03:32:01.811129+0000","last_fullsized":"2024-09-22T03:32:01.811129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1022,"num_write_kb":7680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'926","reported_seq":963,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.587674+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:32:01.587674+0000","last_peered":"2024-09-22T03:32:01.587674+0000","last_clean":"2024-09-22T03:32:01.587674+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:32:01.587674+0000","last_undegraded":"2024-09-22T03:32:01.587674+0000","last_fullsized":"2024-09-22T03:32:01.587674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":927,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'811","reported_seq":848,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.767045+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:32:01.767045+0000","last_peered":"2024-09-22T03:32:01.767045+0000","last_clean":"2024-09-22T03:32:01.767045+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:32:01.767045+0000","last_undegraded":"2024-09-22T03:32:01.767045+0000","last_fullsized":"2024-09-22T03:32:01.767045+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":812,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'866","reported_seq":903,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.876832+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:32:01.876832+0000","last_peered":"2024-09-22T03:32:01.876832+0000","last_clean":"2024-09-22T03:32:01.876832+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:32:01.876832+0000","last_undegraded":"2024-09-22T03:32:01.876832+0000","last_fullsized":"2024-09-22T03:32:01.876832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":867,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'903","reported_seq":940,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.907771+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:32:01.907771+0000","last_peered":"2024-09-22T03:32:01.907771+0000","last_clean":"2024-09-22T03:32:01.907771+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:32:01.907771+0000","last_undegraded":"2024-09-22T03:32:01.907771+0000","last_fullsized":"2024-09-22T03:32:01.907771+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'750","reported_seq":787,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.900012+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:32:01.900012+0000","last_peered":"2024-09-22T03:32:01.900012+0000","last_clean":"2024-09-22T03:32:01.900012+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:32:01.900012+0000","last_undegraded":"2024-09-22T03:32:01.900012+0000","last_fullsized":"2024-09-22T03:32:01.900012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'736","reported_seq":773,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.786754+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:32:01.786754+0000","last_peered":"2024-09-22T03:32:01.786754+0000","last_clean":"2024-09-22T03:32:01.786754+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:32:01.786754+0000","last_undegraded":"2024-09-22T03:32:01.786754+0000","last_fullsized":"2024-09-22T03:32:01.786754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":737,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'920","reported_seq":957,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.781241+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:32:01.781241+0000","last_peered":"2024-09-22T03:32:01.781241+0000","last_clean":"2024-09-22T03:32:01.781241+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:32:01.781241+0000","last_undegraded":"2024-09-22T03:32:01.781241+0000","last_fullsized":"2024-09-22T03:32:01.781241+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'769","reported_seq":806,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.654343+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:32:01.654343+0000","last_peered":"2024-09-22T03:32:01.654343+0000","last_clean":"2024-09-22T03:32:01.654343+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:32:01.654343+0000","last_undegraded":"2024-09-22T03:32:01.654343+0000","last_fullsized":"2024-09-22T03:32:01.654343+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":770,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'1071","reported_seq":1108,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.867734+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:32:01.867734+0000","last_peered":"2024-09-22T03:32:01.867734+0000","last_clean":"2024-09-22T03:32:01.867734+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:32:01.867734+0000","last_undegraded":"2024-09-22T03:32:01.867734+0000","last_fullsized":"2024-09-22T03:32:01.867734+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1072,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'984","reported_seq":1021,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:01.943433+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:32:01.943433+0000","last_peered":"2024-09-22T03:32:01.943433+0000","last_clean":"2024-09-22T03:32:01.943433+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:32:01.943433+0000","last_undegraded":"2024-09-22T03:32:01.943433+0000","last_fullsized":"2024-09-22T03:32:01.943433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":984,"log_dups_size":0,"ondisk_log_size":984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":985,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":56025112,"num_objects":856,"num_object_clones":0,"num_object_copies":2568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13860,"num_write_kb":104633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213838819328,"internally_reserved":0,"allocated":372674560,"data_stored":372674560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13844,"ondisk_log_size":13844,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213838819328,"internally_reserved":0,"allocated":372674560,"data_stored":372674560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53434732544,"internally_reserved":0,"allocated":118140928,"data_stored":118140928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":131480,"kb_used_data":131480,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166248,"statfs":{"total":53552873472,"available":53418237952,"internally_reserved":0,"allocated":134635520,"data_stored":134635520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":74920,"kb_used_data":74920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52222808,"statfs":{"total":53552873472,"available":53476155392,"internally_reserved":0,"allocated":76718080,"data_stored":76718080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":108908,"kb_used_data":108908,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52188820,"statfs":{"total":53552873472,"available":53441351680,"internally_reserved":0,"allocated":111521792,"data_stored":111521792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":97620,"kb_used_data":97620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52200108,"statfs":{"total":53552873472,"available":53452910592,"internally_reserved":0,"allocated":99962880,"data_stored":99962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53434732544,"internally_reserved":0,"allocated":118140928,"data_stored":118140928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53463871488,"internally_reserved":0,"allocated":89001984,"data_stored":89001984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53454659584,"internally_reserved":0,"allocated":98213888,"data_stored":98213888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53485293568,"internally_reserved":0,"allocated":67579904,"data_stored":67579904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53434994688,"internally_reserved":0,"allocated":117878784,"data_stored":117878784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53463871488,"internally_reserved":0,"allocated":89001984,"data_stored":89001984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53454659584,"internally_reserved":0,"allocated":98213888,"data_stored":98213888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53485293568,"internally_reserved":0,"allocated":67579904,"data_stored":67579904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53434994688,"internally_reserved":0,"allocated":117878784,"data_stored":117878784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:32:04.770 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:04.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:04.902+0000 7f62d0997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:04.908+0000 7f62d0997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:07.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:07.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:07.342+0000 7f94cd618640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:07.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:07.348+0000 7f94cd618640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:07.826 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 1635 objects 2024-09-22T03:32:07.826 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :10.5979 2024-09-22T03:32:08.815 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:08.815 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:08.815 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:32:08.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:08.927+0000 7fae74957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:08.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:08.933+0000 7fae74957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:32:09.110 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:32:09.123 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-22T03:32:08.036372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6349387,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":14766,"num_write_kb":106529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14711,"ondisk_log_size":14711,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":112564,"kb_used_data":112564,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209078348,"statfs":{"total":214211493888,"available":214096228352,"internally_reserved":0,"allocated":115265536,"data_stored":115265536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-96624616,"num_objects":-1474,"num_object_clones":0,"num_object_copies":-4422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2129,"num_write_kb":4649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002402"},"pg_stats":[{"pgid":"7.f","version":"76'896","reported_seq":933,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.844622+0000","last_change":"2024-09-22T03:30:25.261909+0000","last_active":"2024-09-22T03:32:06.844622+0000","last_peered":"2024-09-22T03:32:06.844622+0000","last_clean":"2024-09-22T03:32:06.844622+0000","last_became_active":"2024-09-22T03:30:25.237598+0000","last_became_peered":"2024-09-22T03:30:25.237598+0000","last_unstale":"2024-09-22T03:32:06.844622+0000","last_undegraded":"2024-09-22T03:32:06.844622+0000","last_fullsized":"2024-09-22T03:32:06.844622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":897,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"76'754","reported_seq":791,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.811581+0000","last_change":"2024-09-22T03:30:25.219804+0000","last_active":"2024-09-22T03:32:06.811581+0000","last_peered":"2024-09-22T03:32:06.811581+0000","last_clean":"2024-09-22T03:32:06.811581+0000","last_became_active":"2024-09-22T03:30:25.204192+0000","last_became_peered":"2024-09-22T03:30:25.204192+0000","last_unstale":"2024-09-22T03:32:06.811581+0000","last_undegraded":"2024-09-22T03:32:06.811581+0000","last_fullsized":"2024-09-22T03:32:06.811581+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":755,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"76'900","reported_seq":938,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.742869+0000","last_change":"2024-09-22T03:30:25.196674+0000","last_active":"2024-09-22T03:32:06.742869+0000","last_peered":"2024-09-22T03:32:06.742869+0000","last_clean":"2024-09-22T03:32:06.742869+0000","last_became_active":"2024-09-22T03:30:25.188287+0000","last_became_peered":"2024-09-22T03:30:25.188287+0000","last_unstale":"2024-09-22T03:32:06.742869+0000","last_undegraded":"2024-09-22T03:32:06.742869+0000","last_fullsized":"2024-09-22T03:32:06.742869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589848,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":901,"num_write_kb":6465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"76'801","reported_seq":838,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.937671+0000","last_change":"2024-09-22T03:30:25.235347+0000","last_active":"2024-09-22T03:32:06.937671+0000","last_peered":"2024-09-22T03:32:06.937671+0000","last_clean":"2024-09-22T03:32:06.937671+0000","last_became_active":"2024-09-22T03:30:25.231545+0000","last_became_peered":"2024-09-22T03:30:25.231545+0000","last_unstale":"2024-09-22T03:32:06.937671+0000","last_undegraded":"2024-09-22T03:32:06.937671+0000","last_fullsized":"2024-09-22T03:32:06.937671+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":802,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"76'954","reported_seq":991,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.854576+0000","last_change":"2024-09-22T03:30:25.225956+0000","last_active":"2024-09-22T03:32:06.854576+0000","last_peered":"2024-09-22T03:32:06.854576+0000","last_clean":"2024-09-22T03:32:06.854576+0000","last_became_active":"2024-09-22T03:30:25.221256+0000","last_became_peered":"2024-09-22T03:30:25.221256+0000","last_unstale":"2024-09-22T03:32:06.854576+0000","last_undegraded":"2024-09-22T03:32:06.854576+0000","last_fullsized":"2024-09-22T03:32:06.854576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450560,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":955,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"76'1071","reported_seq":1108,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.830250+0000","last_change":"2024-09-22T03:30:25.254862+0000","last_active":"2024-09-22T03:32:06.830250+0000","last_peered":"2024-09-22T03:32:06.830250+0000","last_clean":"2024-09-22T03:32:06.830250+0000","last_became_active":"2024-09-22T03:30:25.251270+0000","last_became_peered":"2024-09-22T03:30:25.251270+0000","last_unstale":"2024-09-22T03:32:06.830250+0000","last_undegraded":"2024-09-22T03:32:06.830250+0000","last_fullsized":"2024-09-22T03:32:06.830250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1072,"num_write_kb":7680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"76'975","reported_seq":1012,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.797557+0000","last_change":"2024-09-22T03:30:25.285841+0000","last_active":"2024-09-22T03:32:06.797557+0000","last_peered":"2024-09-22T03:32:06.797557+0000","last_clean":"2024-09-22T03:32:06.797557+0000","last_became_active":"2024-09-22T03:30:25.272921+0000","last_became_peered":"2024-09-22T03:30:25.272921+0000","last_unstale":"2024-09-22T03:32:06.797557+0000","last_undegraded":"2024-09-22T03:32:06.797557+0000","last_fullsized":"2024-09-22T03:32:06.797557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"76'859","reported_seq":896,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.958162+0000","last_change":"2024-09-22T03:30:25.268591+0000","last_active":"2024-09-22T03:32:06.958162+0000","last_peered":"2024-09-22T03:32:06.958162+0000","last_clean":"2024-09-22T03:32:06.958162+0000","last_became_active":"2024-09-22T03:30:25.262632+0000","last_became_peered":"2024-09-22T03:30:25.262632+0000","last_unstale":"2024-09-22T03:32:06.958162+0000","last_undegraded":"2024-09-22T03:32:06.958162+0000","last_fullsized":"2024-09-22T03:32:06.958162+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":860,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"76'916","reported_seq":953,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.892352+0000","last_change":"2024-09-22T03:30:25.290220+0000","last_active":"2024-09-22T03:32:06.892352+0000","last_peered":"2024-09-22T03:32:06.892352+0000","last_clean":"2024-09-22T03:32:06.892352+0000","last_became_active":"2024-09-22T03:30:25.284617+0000","last_became_peered":"2024-09-22T03:30:25.284617+0000","last_unstale":"2024-09-22T03:32:06.892352+0000","last_undegraded":"2024-09-22T03:32:06.892352+0000","last_fullsized":"2024-09-22T03:32:06.892352+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":917,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":331,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.778622+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:30:45.778622+0000","last_peered":"2024-09-22T03:30:45.778622+0000","last_clean":"2024-09-22T03:30:45.778622+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:30:45.778622+0000","last_undegraded":"2024-09-22T03:30:45.778622+0000","last_fullsized":"2024-09-22T03:30:45.778622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.773969+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:30:45.773969+0000","last_peered":"2024-09-22T03:30:45.773969+0000","last_clean":"2024-09-22T03:30:45.773969+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:30:45.773969+0000","last_undegraded":"2024-09-22T03:30:45.773969+0000","last_fullsized":"2024-09-22T03:30:45.773969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"76'949","reported_seq":986,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.721967+0000","last_change":"2024-09-22T03:30:25.251931+0000","last_active":"2024-09-22T03:32:06.721967+0000","last_peered":"2024-09-22T03:32:06.721967+0000","last_clean":"2024-09-22T03:32:06.721967+0000","last_became_active":"2024-09-22T03:30:25.222329+0000","last_became_peered":"2024-09-22T03:30:25.222329+0000","last_unstale":"2024-09-22T03:32:06.721967+0000","last_undegraded":"2024-09-22T03:32:06.721967+0000","last_fullsized":"2024-09-22T03:32:06.721967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":950,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.767517+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:30:45.767517+0000","last_peered":"2024-09-22T03:30:45.767517+0000","last_clean":"2024-09-22T03:30:45.767517+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:30:45.767517+0000","last_undegraded":"2024-09-22T03:30:45.767517+0000","last_fullsized":"2024-09-22T03:30:45.767517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"76'798","reported_seq":835,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.713735+0000","last_change":"2024-09-22T03:30:25.254359+0000","last_active":"2024-09-22T03:32:06.713735+0000","last_peered":"2024-09-22T03:32:06.713735+0000","last_clean":"2024-09-22T03:32:06.713735+0000","last_became_active":"2024-09-22T03:30:25.236552+0000","last_became_peered":"2024-09-22T03:30:25.236552+0000","last_unstale":"2024-09-22T03:32:06.713735+0000","last_undegraded":"2024-09-22T03:32:06.713735+0000","last_fullsized":"2024-09-22T03:32:06.713735+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":799,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.770047+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.770047+0000","last_peered":"2024-09-22T03:30:45.770047+0000","last_clean":"2024-09-22T03:30:45.770047+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:30:45.770047+0000","last_undegraded":"2024-09-22T03:30:45.770047+0000","last_fullsized":"2024-09-22T03:30:45.770047+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"76'779","reported_seq":816,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.827076+0000","last_change":"2024-09-22T03:30:25.304180+0000","last_active":"2024-09-22T03:32:06.827076+0000","last_peered":"2024-09-22T03:32:06.827076+0000","last_clean":"2024-09-22T03:32:06.827076+0000","last_became_active":"2024-09-22T03:30:25.298546+0000","last_became_peered":"2024-09-22T03:30:25.298546+0000","last_unstale":"2024-09-22T03:32:06.827076+0000","last_undegraded":"2024-09-22T03:32:06.827076+0000","last_fullsized":"2024-09-22T03:32:06.827076+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":780,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":198,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.762920+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:30:45.762920+0000","last_peered":"2024-09-22T03:30:45.762920+0000","last_clean":"2024-09-22T03:30:45.762920+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:30:45.762920+0000","last_undegraded":"2024-09-22T03:30:45.762920+0000","last_fullsized":"2024-09-22T03:30:45.762920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"76'974","reported_seq":1011,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.800569+0000","last_change":"2024-09-22T03:30:25.210137+0000","last_active":"2024-09-22T03:32:06.800569+0000","last_peered":"2024-09-22T03:32:06.800569+0000","last_clean":"2024-09-22T03:32:06.800569+0000","last_became_active":"2024-09-22T03:30:25.196470+0000","last_became_peered":"2024-09-22T03:30:25.196470+0000","last_unstale":"2024-09-22T03:32:06.800569+0000","last_undegraded":"2024-09-22T03:32:06.800569+0000","last_fullsized":"2024-09-22T03:32:06.800569+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":975,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.782456+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.782456+0000","last_peered":"2024-09-22T03:30:45.782456+0000","last_clean":"2024-09-22T03:30:45.782456+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:30:45.782456+0000","last_undegraded":"2024-09-22T03:30:45.782456+0000","last_fullsized":"2024-09-22T03:30:45.782456+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"76'813","reported_seq":850,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.937682+0000","last_change":"2024-09-22T03:30:25.267474+0000","last_active":"2024-09-22T03:32:06.937682+0000","last_peered":"2024-09-22T03:32:06.937682+0000","last_clean":"2024-09-22T03:32:06.937682+0000","last_became_active":"2024-09-22T03:30:25.259393+0000","last_became_peered":"2024-09-22T03:30:25.259393+0000","last_unstale":"2024-09-22T03:32:06.937682+0000","last_undegraded":"2024-09-22T03:32:06.937682+0000","last_fullsized":"2024-09-22T03:32:06.937682+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":814,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780327+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:30:45.780327+0000","last_peered":"2024-09-22T03:30:45.780327+0000","last_clean":"2024-09-22T03:30:45.780327+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:30:45.780327+0000","last_undegraded":"2024-09-22T03:30:45.780327+0000","last_fullsized":"2024-09-22T03:30:45.780327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"76'1128","reported_seq":1165,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.621392+0000","last_change":"2024-09-22T03:30:25.233297+0000","last_active":"2024-09-22T03:32:06.621392+0000","last_peered":"2024-09-22T03:32:06.621392+0000","last_clean":"2024-09-22T03:32:06.621392+0000","last_became_active":"2024-09-22T03:30:25.212925+0000","last_became_peered":"2024-09-22T03:30:25.212925+0000","last_unstale":"2024-09-22T03:32:06.621392+0000","last_undegraded":"2024-09-22T03:32:06.621392+0000","last_fullsized":"2024-09-22T03:32:06.621392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1129,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.780252+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:30:45.780252+0000","last_peered":"2024-09-22T03:30:45.780252+0000","last_clean":"2024-09-22T03:30:45.780252+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:30:45.780252+0000","last_undegraded":"2024-09-22T03:30:45.780252+0000","last_fullsized":"2024-09-22T03:30:45.780252+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"76'1044","reported_seq":1081,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:32:06.675451+0000","last_change":"2024-09-22T03:30:25.284062+0000","last_active":"2024-09-22T03:32:06.675451+0000","last_peered":"2024-09-22T03:32:06.675451+0000","last_clean":"2024-09-22T03:32:06.675451+0000","last_became_active":"2024-09-22T03:30:25.275483+0000","last_became_peered":"2024-09-22T03:30:25.275483+0000","last_unstale":"2024-09-22T03:32:06.675451+0000","last_undegraded":"2024-09-22T03:32:06.675451+0000","last_fullsized":"2024-09-22T03:32:06.675451+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:30:24.112452+0000","last_clean_scrub_stamp":"2024-09-22T03:30:24.112452+0000","objects_scrubbed":0,"log_size":1044,"log_dups_size":0,"ondisk_log_size":1044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1045,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-22T03:30:45.772543+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:30:45.772543+0000","last_peered":"2024-09-22T03:30:45.772543+0000","last_clean":"2024-09-22T03:30:45.772543+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:30:45.772543+0000","last_undegraded":"2024-09-22T03:30:45.772543+0000","last_fullsized":"2024-09-22T03:30:45.772543+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":5759000,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14627,"num_write_kb":104633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214141149184,"internally_reserved":0,"allocated":70344704,"data_stored":70344704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14611,"ondisk_log_size":14611,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214141018112,"internally_reserved":0,"allocated":70475776,"data_stored":70475776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53529923584,"internally_reserved":0,"allocated":22949888,"data_stored":22949888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":36620,"kb_used_data":36620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52261108,"statfs":{"total":53552873472,"available":53515374592,"internally_reserved":0,"allocated":37498880,"data_stored":37498880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":20700,"kb_used_data":20700,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52277028,"statfs":{"total":53552873472,"available":53531676672,"internally_reserved":0,"allocated":21196800,"data_stored":21196800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":29216,"kb_used_data":29216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52268512,"statfs":{"total":53552873472,"available":53522956288,"internally_reserved":0,"allocated":29917184,"data_stored":29917184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":26028,"kb_used_data":26028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52271700,"statfs":{"total":53552873472,"available":53526220800,"internally_reserved":0,"allocated":26652672,"data_stored":26652672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53529923584,"internally_reserved":0,"allocated":22949888,"data_stored":22949888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53536968704,"internally_reserved":0,"allocated":15904768,"data_stored":15904768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53534625792,"internally_reserved":0,"allocated":18247680,"data_stored":18247680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53539500032,"internally_reserved":0,"allocated":13373440,"data_stored":13373440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53529923584,"internally_reserved":0,"allocated":22949888,"data_stored":22949888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":53552873472,"available":53536968704,"internally_reserved":0,"allocated":15904768,"data_stored":15904768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":53552873472,"available":53534625792,"internally_reserved":0,"allocated":18247680,"data_stored":18247680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":53552873472,"available":53539500032,"internally_reserved":0,"allocated":13373440,"data_stored":13373440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":53552873472,"available":53530054656,"internally_reserved":0,"allocated":22818816,"data_stored":22818816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:32:09.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:09.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:09.756+0000 7fafcb26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:09.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:09.761+0000 7fafcb26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:09.859 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-22T03:32:09.859 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-22T03:32:09.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:09.959+0000 7f7e58ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:09.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:09.970+0000 7f7e58ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:10.770 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_4' removed 2024-09-22T03:32:10.794 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:32:10.794 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:32:10.795 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_5 2024-09-22T03:32:10.795 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_5 16 2024-09-22T03:32:10.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:10.958+0000 7faa5b46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:10.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:10.976+0000 7faa5b46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:11.771 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_5' created 2024-09-22T03:32:11.790 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_5 rados --yes-i-really-mean-it || true 2024-09-22T03:32:11.838 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:11.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:11.933+0000 7ff4ab22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:11.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:11.951+0000 7ff4ab22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:11.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:11.968+0000 7f2aca997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:11.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:11.973+0000 7f2aca997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:12.779 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_5' 2024-09-22T03:32:13.799 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_5 bench 90 write' 2024-09-22T03:32:13.802 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:32:13.802 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:32:13.861 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:32:13.861 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:32:13.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_53510 2024-09-22T03:32:13.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:32:13.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:32:14.047 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:14.125 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:14.125 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:14.126 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:32:14.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:14.204+0000 7fcbf2b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:14.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:14.213+0000 7fcbf2b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:14.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:14.260+0000 7f8abc757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:14.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:14.272+0000 7f8abc757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:14.474 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:32:14.475 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:32:14.492 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-09-22T03:32:14.037703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":8940,"kb_used_data":8940,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209181972,"statfs":{"total":214211493888,"available":214202339328,"internally_reserved":0,"allocated":9154560,"data_stored":9154560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5759000,"num_objects":-89,"num_object_clones":0,"num_object_copies":-267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-14627,"num_write_kb":-104633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000870"},"pg_stats":[{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.806972+0000","last_change":"2024-09-22T03:32:11.806972+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.806972+0000","last_undegraded":"2024-09-22T03:32:11.806972+0000","last_fullsized":"2024-09-22T03:32:11.806972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.794024+0000","last_change":"2024-09-22T03:32:11.794024+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.794024+0000","last_undegraded":"2024-09-22T03:32:11.794024+0000","last_fullsized":"2024-09-22T03:32:11.794024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.839733+0000","last_change":"2024-09-22T03:32:11.839733+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.839733+0000","last_undegraded":"2024-09-22T03:32:11.839733+0000","last_fullsized":"2024-09-22T03:32:11.839733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.809942+0000","last_change":"2024-09-22T03:32:11.809942+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.809942+0000","last_undegraded":"2024-09-22T03:32:11.809942+0000","last_fullsized":"2024-09-22T03:32:11.809942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.822579+0000","last_change":"2024-09-22T03:32:11.822579+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.822579+0000","last_undegraded":"2024-09-22T03:32:11.822579+0000","last_fullsized":"2024-09-22T03:32:11.822579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.797885+0000","last_change":"2024-09-22T03:32:11.797885+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.797885+0000","last_undegraded":"2024-09-22T03:32:11.797885+0000","last_fullsized":"2024-09-22T03:32:11.797885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.799232+0000","last_change":"2024-09-22T03:32:11.799232+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.799232+0000","last_undegraded":"2024-09-22T03:32:11.799232+0000","last_fullsized":"2024-09-22T03:32:11.799232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.805293+0000","last_change":"2024-09-22T03:32:11.805293+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.805293+0000","last_undegraded":"2024-09-22T03:32:11.805293+0000","last_fullsized":"2024-09-22T03:32:11.805293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.833004+0000","last_change":"2024-09-22T03:32:11.833004+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.833004+0000","last_undegraded":"2024-09-22T03:32:11.833004+0000","last_fullsized":"2024-09-22T03:32:11.833004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":337,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.790577+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:11.790577+0000","last_peered":"2024-09-22T03:32:11.790577+0000","last_clean":"2024-09-22T03:32:11.790577+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:11.790577+0000","last_undegraded":"2024-09-22T03:32:11.790577+0000","last_fullsized":"2024-09-22T03:32:11.790577+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":203,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.797442+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:11.797442+0000","last_peered":"2024-09-22T03:32:11.797442+0000","last_clean":"2024-09-22T03:32:11.797442+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:11.797442+0000","last_undegraded":"2024-09-22T03:32:11.797442+0000","last_fullsized":"2024-09-22T03:32:11.797442+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.814455+0000","last_change":"2024-09-22T03:32:11.814455+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.814455+0000","last_undegraded":"2024-09-22T03:32:11.814455+0000","last_fullsized":"2024-09-22T03:32:11.814455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":202,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.793487+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:11.793487+0000","last_peered":"2024-09-22T03:32:11.793487+0000","last_clean":"2024-09-22T03:32:11.793487+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:11.793487+0000","last_undegraded":"2024-09-22T03:32:11.793487+0000","last_fullsized":"2024-09-22T03:32:11.793487+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.852295+0000","last_change":"2024-09-22T03:32:11.852295+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.852295+0000","last_undegraded":"2024-09-22T03:32:11.852295+0000","last_fullsized":"2024-09-22T03:32:11.852295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":202,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.794979+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:11.794979+0000","last_peered":"2024-09-22T03:32:11.794979+0000","last_clean":"2024-09-22T03:32:11.794979+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:11.794979+0000","last_undegraded":"2024-09-22T03:32:11.794979+0000","last_fullsized":"2024-09-22T03:32:11.794979+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.861202+0000","last_change":"2024-09-22T03:32:11.861202+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.861202+0000","last_undegraded":"2024-09-22T03:32:11.861202+0000","last_fullsized":"2024-09-22T03:32:11.861202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":204,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.792127+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:11.792127+0000","last_peered":"2024-09-22T03:32:11.792127+0000","last_clean":"2024-09-22T03:32:11.792127+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:11.792127+0000","last_undegraded":"2024-09-22T03:32:11.792127+0000","last_fullsized":"2024-09-22T03:32:11.792127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.903208+0000","last_change":"2024-09-22T03:32:11.903208+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.903208+0000","last_undegraded":"2024-09-22T03:32:11.903208+0000","last_fullsized":"2024-09-22T03:32:11.903208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":202,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.788547+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:11.788547+0000","last_peered":"2024-09-22T03:32:11.788547+0000","last_clean":"2024-09-22T03:32:11.788547+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:11.788547+0000","last_undegraded":"2024-09-22T03:32:11.788547+0000","last_fullsized":"2024-09-22T03:32:11.788547+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.825978+0000","last_change":"2024-09-22T03:32:11.825978+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.825978+0000","last_undegraded":"2024-09-22T03:32:11.825978+0000","last_fullsized":"2024-09-22T03:32:11.825978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":202,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.792765+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:11.792765+0000","last_peered":"2024-09-22T03:32:11.792765+0000","last_clean":"2024-09-22T03:32:11.792765+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:11.792765+0000","last_undegraded":"2024-09-22T03:32:11.792765+0000","last_fullsized":"2024-09-22T03:32:11.792765+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.866791+0000","last_change":"2024-09-22T03:32:11.866791+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.866791+0000","last_undegraded":"2024-09-22T03:32:11.866791+0000","last_fullsized":"2024-09-22T03:32:11.866791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":202,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.788796+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:11.788796+0000","last_peered":"2024-09-22T03:32:11.788796+0000","last_clean":"2024-09-22T03:32:11.788796+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:11.788796+0000","last_undegraded":"2024-09-22T03:32:11.788796+0000","last_fullsized":"2024-09-22T03:32:11.788796+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2024-09-22T03:32:11.885810+0000","last_change":"2024-09-22T03:32:11.885810+0000","last_active":"2024-09-22T03:32:11.770328+0000","last_peered":"2024-09-22T03:32:11.770328+0000","last_clean":"2024-09-22T03:32:11.770328+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:32:11.885810+0000","last_undegraded":"2024-09-22T03:32:11.885810+0000","last_fullsized":"2024-09-22T03:32:11.885810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":202,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-22T03:32:11.797269+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:11.797269+0000","last_peered":"2024-09-22T03:32:11.797269+0000","last_clean":"2024-09-22T03:32:11.797269+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:11.797269+0000","last_undegraded":"2024-09-22T03:32:11.797269+0000","last_fullsized":"2024-09-22T03:32:11.797269+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214201278464,"internally_reserved":0,"allocated":10215424,"data_stored":10215424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214201278464,"internally_reserved":0,"allocated":10215424,"data_stored":10215424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550321664,"internally_reserved":0,"allocated":2551808,"data_stored":2551808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706213,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2304,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295424,"statfs":{"total":53552873472,"available":53550514176,"internally_reserved":0,"allocated":2359296,"data_stored":2359296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706213,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2552,"kb_used_data":2552,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295176,"statfs":{"total":53552873472,"available":53550260224,"internally_reserved":0,"allocated":2613248,"data_stored":2613248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706213,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1432,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296296,"statfs":{"total":53552873472,"available":53551407104,"internally_reserved":0,"allocated":1466368,"data_stored":1466368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706213,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2652,"kb_used_data":2652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295076,"statfs":{"total":53552873472,"available":53550157824,"internally_reserved":0,"allocated":2715648,"data_stored":2715648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550321664,"internally_reserved":0,"allocated":2551808,"data_stored":2551808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53549699072,"internally_reserved":0,"allocated":3174400,"data_stored":3174400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551136768,"internally_reserved":0,"allocated":1736704,"data_stored":1736704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550120960,"internally_reserved":0,"allocated":2752512,"data_stored":2752512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550321664,"internally_reserved":0,"allocated":2551808,"data_stored":2551808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53549699072,"internally_reserved":0,"allocated":3174400,"data_stored":3174400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53551136768,"internally_reserved":0,"allocated":1736704,"data_stored":1736704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53550120960,"internally_reserved":0,"allocated":2752512,"data_stored":2752512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53550321664,"internally_reserved":0,"allocated":2551808,"data_stored":2551808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:32:14.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 132 116 0.906197 0.90625 0.15555 0.125091 2024-09-22T03:32:15.861 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 275 259 1.01163 1.11719 0.0324931 0.116309 2024-09-22T03:32:16.459 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:16.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:16.582+0000 7f8f6a90e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:16.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:16.589+0000 7f8f6a90e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:16.861 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 443 427 1.11188 1.3125 0.210909 0.109923 2024-09-22T03:32:17.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 607 591 1.1542 1.28125 0.131513 0.107024 2024-09-22T03:32:18.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:18.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 769 753 1.17646 1.26562 0.119758 0.104874 2024-09-22T03:32:18.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:18.922+0000 7f72ee66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:18.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:18.928+0000 7f72ee66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:19.494 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:19.494 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:19.494 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:32:19.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:19.628+0000 7fc6d186c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:19.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:19.639+0000 7fc6d186c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:19.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 897 881 1.14704 1 0.142337 0.106709 2024-09-22T03:32:19.897 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:32:19.897 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:32:19.914 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-22T03:32:18.038326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4416051,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":606,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":27168,"kb_used_data":27168,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209163744,"statfs":{"total":214211493888,"available":214183673856,"internally_reserved":0,"allocated":27820032,"data_stored":27820032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3825664,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":3736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.266531"},"pg_stats":[{"pgid":"8.7","version":"80'15","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.541965+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:32:16.541965+0000","last_peered":"2024-09-22T03:32:16.541965+0000","last_clean":"2024-09-22T03:32:16.541965+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:32:16.541965+0000","last_undegraded":"2024-09-22T03:32:16.541965+0000","last_fullsized":"2024-09-22T03:32:16.541965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"80'39","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.380421+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:32:16.380421+0000","last_peered":"2024-09-22T03:32:16.380421+0000","last_clean":"2024-09-22T03:32:16.380421+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:32:16.380421+0000","last_undegraded":"2024-09-22T03:32:16.380421+0000","last_fullsized":"2024-09-22T03:32:16.380421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"80'31","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.671877+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:32:16.671877+0000","last_peered":"2024-09-22T03:32:16.671877+0000","last_clean":"2024-09-22T03:32:16.671877+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:32:16.671877+0000","last_undegraded":"2024-09-22T03:32:16.671877+0000","last_fullsized":"2024-09-22T03:32:16.671877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"80'23","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.971631+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:32:16.971631+0000","last_peered":"2024-09-22T03:32:16.971631+0000","last_clean":"2024-09-22T03:32:16.971631+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:32:16.971631+0000","last_undegraded":"2024-09-22T03:32:16.971631+0000","last_fullsized":"2024-09-22T03:32:16.971631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.817198+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:32:13.817198+0000","last_peered":"2024-09-22T03:32:13.817198+0000","last_clean":"2024-09-22T03:32:13.817198+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:32:13.817198+0000","last_undegraded":"2024-09-22T03:32:13.817198+0000","last_fullsized":"2024-09-22T03:32:13.817198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"80'15","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.447565+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:32:16.447565+0000","last_peered":"2024-09-22T03:32:16.447565+0000","last_clean":"2024-09-22T03:32:16.447565+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:32:16.447565+0000","last_undegraded":"2024-09-22T03:32:16.447565+0000","last_fullsized":"2024-09-22T03:32:16.447565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"80'39","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.898677+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:32:16.898677+0000","last_peered":"2024-09-22T03:32:16.898677+0000","last_clean":"2024-09-22T03:32:16.898677+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:32:16.898677+0000","last_undegraded":"2024-09-22T03:32:16.898677+0000","last_fullsized":"2024-09-22T03:32:16.898677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"80'15","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.927582+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:32:16.927582+0000","last_peered":"2024-09-22T03:32:16.927582+0000","last_clean":"2024-09-22T03:32:16.927582+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:32:16.927582+0000","last_undegraded":"2024-09-22T03:32:16.927582+0000","last_fullsized":"2024-09-22T03:32:16.927582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"80'31","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.304080+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:32:16.304080+0000","last_peered":"2024-09-22T03:32:16.304080+0000","last_clean":"2024-09-22T03:32:16.304080+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:32:16.304080+0000","last_undegraded":"2024-09-22T03:32:16.304080+0000","last_fullsized":"2024-09-22T03:32:16.304080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":343,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.804356+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:13.804356+0000","last_peered":"2024-09-22T03:32:13.804356+0000","last_clean":"2024-09-22T03:32:13.804356+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:13.804356+0000","last_undegraded":"2024-09-22T03:32:13.804356+0000","last_fullsized":"2024-09-22T03:32:13.804356+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":209,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.818739+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:13.818739+0000","last_peered":"2024-09-22T03:32:13.818739+0000","last_clean":"2024-09-22T03:32:13.818739+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:13.818739+0000","last_undegraded":"2024-09-22T03:32:13.818739+0000","last_fullsized":"2024-09-22T03:32:13.818739+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"80'10","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.979676+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:32:16.979676+0000","last_peered":"2024-09-22T03:32:16.979676+0000","last_clean":"2024-09-22T03:32:16.979676+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:32:16.979676+0000","last_undegraded":"2024-09-22T03:32:16.979676+0000","last_fullsized":"2024-09-22T03:32:16.979676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90112,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808550+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:13.808550+0000","last_peered":"2024-09-22T03:32:13.808550+0000","last_clean":"2024-09-22T03:32:13.808550+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:13.808550+0000","last_undegraded":"2024-09-22T03:32:13.808550+0000","last_fullsized":"2024-09-22T03:32:13.808550+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"80'39","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.847780+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:32:16.847780+0000","last_peered":"2024-09-22T03:32:16.847780+0000","last_clean":"2024-09-22T03:32:16.847780+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:32:16.847780+0000","last_undegraded":"2024-09-22T03:32:16.847780+0000","last_fullsized":"2024-09-22T03:32:16.847780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.816675+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.816675+0000","last_peered":"2024-09-22T03:32:13.816675+0000","last_clean":"2024-09-22T03:32:13.816675+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:13.816675+0000","last_undegraded":"2024-09-22T03:32:13.816675+0000","last_fullsized":"2024-09-22T03:32:13.816675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"80'31","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.438361+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:32:16.438361+0000","last_peered":"2024-09-22T03:32:16.438361+0000","last_clean":"2024-09-22T03:32:16.438361+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:32:16.438361+0000","last_undegraded":"2024-09-22T03:32:16.438361+0000","last_fullsized":"2024-09-22T03:32:16.438361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":210,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.805696+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.805696+0000","last_peered":"2024-09-22T03:32:13.805696+0000","last_clean":"2024-09-22T03:32:13.805696+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:13.805696+0000","last_undegraded":"2024-09-22T03:32:13.805696+0000","last_fullsized":"2024-09-22T03:32:13.805696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"80'47","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.614459+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:32:16.614459+0000","last_peered":"2024-09-22T03:32:16.614459+0000","last_clean":"2024-09-22T03:32:16.614459+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:32:16.614459+0000","last_undegraded":"2024-09-22T03:32:16.614459+0000","last_fullsized":"2024-09-22T03:32:16.614459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814199+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.814199+0000","last_peered":"2024-09-22T03:32:13.814199+0000","last_clean":"2024-09-22T03:32:13.814199+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:13.814199+0000","last_undegraded":"2024-09-22T03:32:13.814199+0000","last_fullsized":"2024-09-22T03:32:13.814199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"80'39","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.794688+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:32:16.794688+0000","last_peered":"2024-09-22T03:32:16.794688+0000","last_clean":"2024-09-22T03:32:16.794688+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:32:16.794688+0000","last_undegraded":"2024-09-22T03:32:16.794688+0000","last_fullsized":"2024-09-22T03:32:16.794688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808280+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.808280+0000","last_peered":"2024-09-22T03:32:13.808280+0000","last_clean":"2024-09-22T03:32:13.808280+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:13.808280+0000","last_undegraded":"2024-09-22T03:32:13.808280+0000","last_fullsized":"2024-09-22T03:32:13.808280+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"80'39","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.347950+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:32:16.347950+0000","last_peered":"2024-09-22T03:32:16.347950+0000","last_clean":"2024-09-22T03:32:16.347950+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:32:16.347950+0000","last_undegraded":"2024-09-22T03:32:16.347950+0000","last_fullsized":"2024-09-22T03:32:16.347950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814132+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:13.814132+0000","last_peered":"2024-09-22T03:32:13.814132+0000","last_clean":"2024-09-22T03:32:13.814132+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:13.814132+0000","last_undegraded":"2024-09-22T03:32:13.814132+0000","last_fullsized":"2024-09-22T03:32:13.814132+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"80'39","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:16.720635+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:32:16.720635+0000","last_peered":"2024-09-22T03:32:16.720635+0000","last_clean":"2024-09-22T03:32:16.720635+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:32:16.720635+0000","last_undegraded":"2024-09-22T03:32:16.720635+0000","last_fullsized":"2024-09-22T03:32:16.720635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.820495+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:13.820495+0000","last_peered":"2024-09-22T03:32:13.820495+0000","last_clean":"2024-09-22T03:32:13.820495+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:13.820495+0000","last_undegraded":"2024-09-22T03:32:13.820495+0000","last_fullsized":"2024-09-22T03:32:13.820495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3825664,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":3736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214176391168,"internally_reserved":0,"allocated":35102720,"data_stored":35102720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":452,"ondisk_log_size":452,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214176391168,"internally_reserved":0,"allocated":35102720,"data_stored":35102720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53544681472,"internally_reserved":0,"allocated":8192000,"data_stored":8192000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6268,"kb_used_data":6268,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52291460,"statfs":{"total":53552873472,"available":53546455040,"internally_reserved":0,"allocated":6418432,"data_stored":6418432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6080,"kb_used_data":6080,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52291648,"statfs":{"total":53552873472,"available":53546647552,"internally_reserved":0,"allocated":6225920,"data_stored":6225920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7076,"kb_used_data":7076,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290652,"statfs":{"total":53552873472,"available":53545627648,"internally_reserved":0,"allocated":7245824,"data_stored":7245824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7744,"kb_used_data":7744,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289984,"statfs":{"total":53552873472,"available":53544943616,"internally_reserved":0,"allocated":7929856,"data_stored":7929856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53544681472,"internally_reserved":0,"allocated":8192000,"data_stored":8192000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53542875136,"internally_reserved":0,"allocated":9998336,"data_stored":9998336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53543624704,"internally_reserved":0,"allocated":9248768,"data_stored":9248768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53545209856,"internally_reserved":0,"allocated":7663616,"data_stored":7663616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53544681472,"internally_reserved":0,"allocated":8192000,"data_stored":8192000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53542875136,"internally_reserved":0,"allocated":9998336,"data_stored":9998336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53543624704,"internally_reserved":0,"allocated":9248768,"data_stored":9248768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53545209856,"internally_reserved":0,"allocated":7663616,"data_stored":7663616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53544681472,"internally_reserved":0,"allocated":8192000,"data_stored":8192000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:32:20.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 1077 1061 1.18405 1.40625 0.0384637 0.105082 2024-09-22T03:32:21.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:21.258+0000 7fa891e18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:21.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:21.264+0000 7fa891e18640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:21.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 16 1240 1224 1.19521 1.27344 0.153734 0.103688 2024-09-22T03:32:22.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1359 1343 1.16569 0.929688 0.0223447 0.104617 2024-09-22T03:32:23.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:23.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:23.625+0000 7f3563d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:23.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:23.632+0000 7f3563d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:23.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 1502 1486 1.16083 1.11719 0.047103 0.106542 2024-09-22T03:32:24.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 1649 1633 1.1597 1.14844 0.0500254 0.106688 2024-09-22T03:32:24.916 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:24.916 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:24.916 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:32:24.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:32:25.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.036+0000 7fceeb4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.045+0000 7fceeb4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.062 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:25.062 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:25.063 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:25.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:25.066 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:25.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:32:25.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.182+0000 7f3bc2d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.191+0000 7f3bc2d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.208 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:25.212 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:25.212 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:25.212 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:32:25.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.325+0000 7f2131197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.334+0000 7f2131197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.348 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:25.348 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:25.348 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:25.348 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:25.349 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:25.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:25.352 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:25.352 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:32:25.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.465+0000 7fcb6266c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.475+0000 7fcb6266c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:25.552 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:25.553 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:25.553 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:25.553 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:25.553 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:25.556 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:25.557 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:25.836 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:25.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 1791 1775 1.15549 1.10938 0.0551731 0.107749 2024-09-22T03:32:25.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.984+0000 7f0549a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:25.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:25.991+0000 7f0549a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:26.862 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 1943 1927 1.15795 1.1875 0.135122 0.106952 2024-09-22T03:32:27.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 2106 2090 1.16619 1.27344 0.144107 0.10678 2024-09-22T03:32:28.250 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:28.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:28.365+0000 7f9bd9f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:28.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:28.370+0000 7f9bd9f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:28.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 2257 2241 1.16708 1.17969 0.234882 0.106815 2024-09-22T03:32:29.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 2406 2390 1.16688 1.16406 0.138782 0.106466 2024-09-22T03:32:30.557 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:30.558 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:30.558 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:32:30.558 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:32:30.634 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:30.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:30.691+0000 7f4227423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:30.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:30.702+0000 7f4227423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:30.773 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:30.774 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:30.778 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:30.778 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:30.778 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:32:30.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:30.801+0000 7fc160e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:30.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:30.808+0000 7fc160e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:30.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 2553 2537 1.16579 1.14844 0.187204 0.106811 2024-09-22T03:32:30.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:30.926+0000 7fbe55a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:30.940+0000 7fbe55a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:30.997 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:30.998 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:30.998 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:31.004 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:31.004 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:31.004 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:32:31.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:31.224+0000 7f9209557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:31.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:31.236+0000 7f9209557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:31.268 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:31.268 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:31.268 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:31.268 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:31.268 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:31.268 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:31.268 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:31.269 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:31.269 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:31.269 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:31.269 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:31.269 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:31.271 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:31.272 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:31.272 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:32:31.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:31.382+0000 7f544d957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:31.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:31.392+0000 7f544d957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:31.407 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:31.407 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:31.407 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:31.407 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:31.407 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:31.407 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:31.408 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:31.408 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:31.408 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:31.408 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:31.408 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:31.408 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:31.410 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:31.410 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:31.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 2707 2691 1.16786 1.20312 0.0844036 0.106626 2024-09-22T03:32:32.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 2868 2852 1.17259 1.25781 0.0874218 0.106149 2024-09-22T03:32:33.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:33.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:33.146+0000 7fdb0846c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:33.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:33.152+0000 7fdb0846c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:33.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:32:33.863050+0000 min lat: 0.0140532 max lat: 0.60614 avg lat: 0.105462 2024-09-22T03:32:33.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:32:33.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 3042 3026 1.18192 1.35938 0.0539949 0.105462 2024-09-22T03:32:34.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 3197 3181 1.1833 1.21094 0.182823 0.105414 2024-09-22T03:32:35.305 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:35.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:35.427+0000 7f8c4f058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:35.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:35.433+0000 7f8c4f058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:35.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 3364 3348 1.18881 1.30469 0.069417 0.104893 2024-09-22T03:32:36.412 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:36.412 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:36.412 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:32:36.412 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:32:36.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:36.551+0000 7f3328997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:36.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:36.566+0000 7f3328997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:36.630 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:36.631 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:36.634 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:36.634 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:36.635 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:32:36.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:36.752+0000 7f0192997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:36.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:36.762+0000 7f0192997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:36.783 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:36.784 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:36.784 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:36.784 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:36.787 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:36.787 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:36.787 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:32:36.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 3515 3499 1.18841 1.17969 0.064746 0.104872 2024-09-22T03:32:36.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:36.943+0000 7f64ff717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:36.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:36.956+0000 7f64ff717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:36.979 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:36.979 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:36.979 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:36.979 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:36.979 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:36.979 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:36.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:36.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:36.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:36.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:36.980 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:36.980 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:36.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:36.984 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:36.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:32:37.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:37.149+0000 7f1862557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:37.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:37.158+0000 7f1862557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:37.180 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:37.181 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:37.183 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:37.184 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:37.662 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:37.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:37.792+0000 7fbed6797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:37.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:37.799+0000 7fbed6797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:37.863 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 3671 3655 1.18967 1.21875 0.110516 0.1048 2024-09-22T03:32:38.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 3816 3800 1.18739 1.13281 0.181943 0.104938 2024-09-22T03:32:39.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 3972 3956 1.18859 1.21875 0.0798447 0.10489 2024-09-22T03:32:40.029 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:40.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:40.143+0000 7f326f917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:40.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:40.151+0000 7f326f917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:40.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 4124 4108 1.18855 1.1875 0.130387 0.104823 2024-09-22T03:32:41.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 4266 4250 1.18571 1.10938 0.114673 0.105125 2024-09-22T03:32:42.185 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:42.185 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:32:42.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:42.293+0000 7f2e08997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:42.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:42.303+0000 7f2e08997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:42.315 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:42.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:42.452+0000 7f103ac2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:42.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:42.460+0000 7f103ac2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:42.768 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:32:42.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:32:42.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 4398 4382 1.18038 1.03125 0.137066 0.105711 2024-09-22T03:32:42.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:42.910+0000 7fcdf64d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:42.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:42.921+0000 7fcdf64d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:43.774 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:32:43.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:32:43.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 4549 4533 1.18036 1.17969 0.0481683 0.105655 2024-09-22T03:32:43.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:43.964+0000 7f4c6cc2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:43.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:43.982+0000 7f4c6cc2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:44.665 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:44.782 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:32:44.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:32:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:44.821+0000 7faa2ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:44.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:44.827+0000 7faa2ef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:44.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 4691 4675 1.17806 1.10938 0.150311 0.10578 2024-09-22T03:32:44.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:44.922+0000 7f1579058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:44.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:44.940+0000 7f1579058640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:32:45.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:32:45.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:32:45.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 16 4830 4814 1.17518 1.08594 0.174613 0.106108 2024-09-22T03:32:45.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:45.887+0000 7fbf0766c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:45.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:45.895+0000 7fbf0766c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:46.126 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:32:46.126 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:32:46.140 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-22T03:32:46.043470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35660339,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4420,"num_write_kb":36144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4365,"ondisk_log_size":4365,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":215288,"kb_used_data":215288,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208975624,"statfs":{"total":214211493888,"available":213991038976,"internally_reserved":0,"allocated":220454912,"data_stored":220454912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000643"},"pg_stats":[{"pgid":"8.7","version":"80'335","reported_seq":348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.847472+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:32:41.847472+0000","last_peered":"2024-09-22T03:32:41.847472+0000","last_clean":"2024-09-22T03:32:41.847472+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:32:41.847472+0000","last_undegraded":"2024-09-22T03:32:41.847472+0000","last_fullsized":"2024-09-22T03:32:41.847472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.672573+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:32:41.672573+0000","last_peered":"2024-09-22T03:32:41.672573+0000","last_clean":"2024-09-22T03:32:41.672573+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:32:41.672573+0000","last_undegraded":"2024-09-22T03:32:41.672573+0000","last_fullsized":"2024-09-22T03:32:41.672573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.790364+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:32:41.790364+0000","last_peered":"2024-09-22T03:32:41.790364+0000","last_clean":"2024-09-22T03:32:41.790364+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:32:41.790364+0000","last_undegraded":"2024-09-22T03:32:41.790364+0000","last_fullsized":"2024-09-22T03:32:41.790364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"80'247","reported_seq":260,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.896832+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:32:41.896832+0000","last_peered":"2024-09-22T03:32:41.896832+0000","last_clean":"2024-09-22T03:32:41.896832+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:32:41.896832+0000","last_undegraded":"2024-09-22T03:32:41.896832+0000","last_fullsized":"2024-09-22T03:32:41.896832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"80'175","reported_seq":188,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.115878+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:32:41.115878+0000","last_peered":"2024-09-22T03:32:41.115878+0000","last_clean":"2024-09-22T03:32:41.115878+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:32:41.115878+0000","last_undegraded":"2024-09-22T03:32:41.115878+0000","last_fullsized":"2024-09-22T03:32:41.115878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"80'247","reported_seq":260,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.926663+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:32:40.926663+0000","last_peered":"2024-09-22T03:32:40.926663+0000","last_clean":"2024-09-22T03:32:40.926663+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:32:40.926663+0000","last_undegraded":"2024-09-22T03:32:40.926663+0000","last_fullsized":"2024-09-22T03:32:40.926663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"80'335","reported_seq":348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.310594+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:32:41.310594+0000","last_peered":"2024-09-22T03:32:41.310594+0000","last_clean":"2024-09-22T03:32:41.310594+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:32:41.310594+0000","last_undegraded":"2024-09-22T03:32:41.310594+0000","last_fullsized":"2024-09-22T03:32:41.310594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"80'271","reported_seq":284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.630635+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:32:41.630635+0000","last_peered":"2024-09-22T03:32:41.630635+0000","last_clean":"2024-09-22T03:32:41.630635+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:32:41.630635+0000","last_undegraded":"2024-09-22T03:32:41.630635+0000","last_fullsized":"2024-09-22T03:32:41.630635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.436981+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:32:41.436981+0000","last_peered":"2024-09-22T03:32:41.436981+0000","last_clean":"2024-09-22T03:32:41.436981+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:32:41.436981+0000","last_undegraded":"2024-09-22T03:32:41.436981+0000","last_fullsized":"2024-09-22T03:32:41.436981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":343,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.804356+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:13.804356+0000","last_peered":"2024-09-22T03:32:13.804356+0000","last_clean":"2024-09-22T03:32:13.804356+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:13.804356+0000","last_undegraded":"2024-09-22T03:32:13.804356+0000","last_fullsized":"2024-09-22T03:32:13.804356+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":209,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.818739+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:13.818739+0000","last_peered":"2024-09-22T03:32:13.818739+0000","last_clean":"2024-09-22T03:32:13.818739+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:13.818739+0000","last_undegraded":"2024-09-22T03:32:13.818739+0000","last_fullsized":"2024-09-22T03:32:13.818739+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"80'271","reported_seq":284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.942163+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:32:41.942163+0000","last_peered":"2024-09-22T03:32:41.942163+0000","last_clean":"2024-09-22T03:32:41.942163+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:32:41.942163+0000","last_undegraded":"2024-09-22T03:32:41.942163+0000","last_fullsized":"2024-09-22T03:32:41.942163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808550+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:13.808550+0000","last_peered":"2024-09-22T03:32:13.808550+0000","last_clean":"2024-09-22T03:32:13.808550+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:13.808550+0000","last_undegraded":"2024-09-22T03:32:13.808550+0000","last_fullsized":"2024-09-22T03:32:13.808550+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.743950+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:32:40.743950+0000","last_peered":"2024-09-22T03:32:40.743950+0000","last_clean":"2024-09-22T03:32:40.743950+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:32:40.743950+0000","last_undegraded":"2024-09-22T03:32:40.743950+0000","last_fullsized":"2024-09-22T03:32:40.743950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.816675+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.816675+0000","last_peered":"2024-09-22T03:32:13.816675+0000","last_clean":"2024-09-22T03:32:13.816675+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:13.816675+0000","last_undegraded":"2024-09-22T03:32:13.816675+0000","last_fullsized":"2024-09-22T03:32:13.816675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"80'223","reported_seq":236,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.832748+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:32:40.832748+0000","last_peered":"2024-09-22T03:32:40.832748+0000","last_clean":"2024-09-22T03:32:40.832748+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:32:40.832748+0000","last_undegraded":"2024-09-22T03:32:40.832748+0000","last_fullsized":"2024-09-22T03:32:40.832748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":210,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.805696+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.805696+0000","last_peered":"2024-09-22T03:32:13.805696+0000","last_clean":"2024-09-22T03:32:13.805696+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:13.805696+0000","last_undegraded":"2024-09-22T03:32:13.805696+0000","last_fullsized":"2024-09-22T03:32:13.805696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.724743+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:32:41.724743+0000","last_peered":"2024-09-22T03:32:41.724743+0000","last_clean":"2024-09-22T03:32:41.724743+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:32:41.724743+0000","last_undegraded":"2024-09-22T03:32:41.724743+0000","last_fullsized":"2024-09-22T03:32:41.724743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814199+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.814199+0000","last_peered":"2024-09-22T03:32:13.814199+0000","last_clean":"2024-09-22T03:32:13.814199+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:13.814199+0000","last_undegraded":"2024-09-22T03:32:13.814199+0000","last_fullsized":"2024-09-22T03:32:13.814199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.539057+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:32:41.539057+0000","last_peered":"2024-09-22T03:32:41.539057+0000","last_clean":"2024-09-22T03:32:41.539057+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:32:41.539057+0000","last_undegraded":"2024-09-22T03:32:41.539057+0000","last_fullsized":"2024-09-22T03:32:41.539057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808280+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.808280+0000","last_peered":"2024-09-22T03:32:13.808280+0000","last_clean":"2024-09-22T03:32:13.808280+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:13.808280+0000","last_undegraded":"2024-09-22T03:32:13.808280+0000","last_fullsized":"2024-09-22T03:32:13.808280+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"80'263","reported_seq":276,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.631509+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:32:40.631509+0000","last_peered":"2024-09-22T03:32:40.631509+0000","last_clean":"2024-09-22T03:32:40.631509+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:32:40.631509+0000","last_undegraded":"2024-09-22T03:32:40.631509+0000","last_fullsized":"2024-09-22T03:32:40.631509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814132+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:13.814132+0000","last_peered":"2024-09-22T03:32:13.814132+0000","last_clean":"2024-09-22T03:32:13.814132+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:13.814132+0000","last_undegraded":"2024-09-22T03:32:13.814132+0000","last_fullsized":"2024-09-22T03:32:13.814132+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"80'272","reported_seq":285,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.371025+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:32:41.371025+0000","last_peered":"2024-09-22T03:32:41.371025+0000","last_clean":"2024-09-22T03:32:41.371025+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:32:41.371025+0000","last_undegraded":"2024-09-22T03:32:41.371025+0000","last_fullsized":"2024-09-22T03:32:41.371025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2236416,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":2184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.820495+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:13.820495+0000","last_peered":"2024-09-22T03:32:13.820495+0000","last_clean":"2024-09-22T03:32:13.820495+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:13.820495+0000","last_undegraded":"2024-09-22T03:32:13.820495+0000","last_fullsized":"2024-09-22T03:32:13.820495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":35069952,"num_objects":536,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4281,"num_write_kb":34248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213984419840,"internally_reserved":0,"allocated":227074048,"data_stored":227074048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4265,"ondisk_log_size":4265,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213984419840,"internally_reserved":0,"allocated":227074048,"data_stored":227074048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":53132,"kb_used_data":53132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52244596,"statfs":{"total":53552873472,"available":53498466304,"internally_reserved":0,"allocated":54407168,"data_stored":54407168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":41948,"kb_used_data":41948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52255780,"statfs":{"total":53552873472,"available":53509918720,"internally_reserved":0,"allocated":42954752,"data_stored":42954752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":62364,"kb_used_data":62364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52235364,"statfs":{"total":53552873472,"available":53489012736,"internally_reserved":0,"allocated":63860736,"data_stored":63860736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":57844,"kb_used_data":57844,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52239884,"statfs":{"total":53552873472,"available":53493641216,"internally_reserved":0,"allocated":59232256,"data_stored":59232256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53492076544,"internally_reserved":0,"allocated":60796928,"data_stored":60796928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53486936064,"internally_reserved":0,"allocated":65937408,"data_stored":65937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53508272128,"internally_reserved":0,"allocated":44601344,"data_stored":44601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53492076544,"internally_reserved":0,"allocated":60796928,"data_stored":60796928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53486936064,"internally_reserved":0,"allocated":65937408,"data_stored":65937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53508272128,"internally_reserved":0,"allocated":44601344,"data_stored":44601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:32:46.141 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:32:46.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:46.254+0000 7ff38082c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:46.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:46.265+0000 7ff38082c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:46.513 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:32:46.514 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:32:46.539 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-22T03:32:46.043470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35660339,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4420,"num_write_kb":36144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4365,"ondisk_log_size":4365,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":215288,"kb_used_data":215288,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208975624,"statfs":{"total":214211493888,"available":213991038976,"internally_reserved":0,"allocated":220454912,"data_stored":220454912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000643"},"pg_stats":[{"pgid":"8.7","version":"80'335","reported_seq":348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.847472+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:32:41.847472+0000","last_peered":"2024-09-22T03:32:41.847472+0000","last_clean":"2024-09-22T03:32:41.847472+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:32:41.847472+0000","last_undegraded":"2024-09-22T03:32:41.847472+0000","last_fullsized":"2024-09-22T03:32:41.847472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.672573+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:32:41.672573+0000","last_peered":"2024-09-22T03:32:41.672573+0000","last_clean":"2024-09-22T03:32:41.672573+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:32:41.672573+0000","last_undegraded":"2024-09-22T03:32:41.672573+0000","last_fullsized":"2024-09-22T03:32:41.672573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.790364+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:32:41.790364+0000","last_peered":"2024-09-22T03:32:41.790364+0000","last_clean":"2024-09-22T03:32:41.790364+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:32:41.790364+0000","last_undegraded":"2024-09-22T03:32:41.790364+0000","last_fullsized":"2024-09-22T03:32:41.790364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"80'247","reported_seq":260,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.896832+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:32:41.896832+0000","last_peered":"2024-09-22T03:32:41.896832+0000","last_clean":"2024-09-22T03:32:41.896832+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:32:41.896832+0000","last_undegraded":"2024-09-22T03:32:41.896832+0000","last_fullsized":"2024-09-22T03:32:41.896832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"80'175","reported_seq":188,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.115878+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:32:41.115878+0000","last_peered":"2024-09-22T03:32:41.115878+0000","last_clean":"2024-09-22T03:32:41.115878+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:32:41.115878+0000","last_undegraded":"2024-09-22T03:32:41.115878+0000","last_fullsized":"2024-09-22T03:32:41.115878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"80'247","reported_seq":260,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.926663+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:32:40.926663+0000","last_peered":"2024-09-22T03:32:40.926663+0000","last_clean":"2024-09-22T03:32:40.926663+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:32:40.926663+0000","last_undegraded":"2024-09-22T03:32:40.926663+0000","last_fullsized":"2024-09-22T03:32:40.926663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"80'335","reported_seq":348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.310594+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:32:41.310594+0000","last_peered":"2024-09-22T03:32:41.310594+0000","last_clean":"2024-09-22T03:32:41.310594+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:32:41.310594+0000","last_undegraded":"2024-09-22T03:32:41.310594+0000","last_fullsized":"2024-09-22T03:32:41.310594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"80'271","reported_seq":284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.630635+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:32:41.630635+0000","last_peered":"2024-09-22T03:32:41.630635+0000","last_clean":"2024-09-22T03:32:41.630635+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:32:41.630635+0000","last_undegraded":"2024-09-22T03:32:41.630635+0000","last_fullsized":"2024-09-22T03:32:41.630635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.436981+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:32:41.436981+0000","last_peered":"2024-09-22T03:32:41.436981+0000","last_clean":"2024-09-22T03:32:41.436981+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:32:41.436981+0000","last_undegraded":"2024-09-22T03:32:41.436981+0000","last_fullsized":"2024-09-22T03:32:41.436981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":343,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.804356+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:13.804356+0000","last_peered":"2024-09-22T03:32:13.804356+0000","last_clean":"2024-09-22T03:32:13.804356+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:13.804356+0000","last_undegraded":"2024-09-22T03:32:13.804356+0000","last_fullsized":"2024-09-22T03:32:13.804356+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":209,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.818739+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:13.818739+0000","last_peered":"2024-09-22T03:32:13.818739+0000","last_clean":"2024-09-22T03:32:13.818739+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:13.818739+0000","last_undegraded":"2024-09-22T03:32:13.818739+0000","last_fullsized":"2024-09-22T03:32:13.818739+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"80'271","reported_seq":284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.942163+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:32:41.942163+0000","last_peered":"2024-09-22T03:32:41.942163+0000","last_clean":"2024-09-22T03:32:41.942163+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:32:41.942163+0000","last_undegraded":"2024-09-22T03:32:41.942163+0000","last_fullsized":"2024-09-22T03:32:41.942163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808550+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:13.808550+0000","last_peered":"2024-09-22T03:32:13.808550+0000","last_clean":"2024-09-22T03:32:13.808550+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:13.808550+0000","last_undegraded":"2024-09-22T03:32:13.808550+0000","last_fullsized":"2024-09-22T03:32:13.808550+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.743950+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:32:40.743950+0000","last_peered":"2024-09-22T03:32:40.743950+0000","last_clean":"2024-09-22T03:32:40.743950+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:32:40.743950+0000","last_undegraded":"2024-09-22T03:32:40.743950+0000","last_fullsized":"2024-09-22T03:32:40.743950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.816675+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.816675+0000","last_peered":"2024-09-22T03:32:13.816675+0000","last_clean":"2024-09-22T03:32:13.816675+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:13.816675+0000","last_undegraded":"2024-09-22T03:32:13.816675+0000","last_fullsized":"2024-09-22T03:32:13.816675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"80'223","reported_seq":236,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.832748+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:32:40.832748+0000","last_peered":"2024-09-22T03:32:40.832748+0000","last_clean":"2024-09-22T03:32:40.832748+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:32:40.832748+0000","last_undegraded":"2024-09-22T03:32:40.832748+0000","last_fullsized":"2024-09-22T03:32:40.832748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":210,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.805696+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.805696+0000","last_peered":"2024-09-22T03:32:13.805696+0000","last_clean":"2024-09-22T03:32:13.805696+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:13.805696+0000","last_undegraded":"2024-09-22T03:32:13.805696+0000","last_fullsized":"2024-09-22T03:32:13.805696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.724743+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:32:41.724743+0000","last_peered":"2024-09-22T03:32:41.724743+0000","last_clean":"2024-09-22T03:32:41.724743+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:32:41.724743+0000","last_undegraded":"2024-09-22T03:32:41.724743+0000","last_fullsized":"2024-09-22T03:32:41.724743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814199+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.814199+0000","last_peered":"2024-09-22T03:32:13.814199+0000","last_clean":"2024-09-22T03:32:13.814199+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:13.814199+0000","last_undegraded":"2024-09-22T03:32:13.814199+0000","last_fullsized":"2024-09-22T03:32:13.814199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.539057+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:32:41.539057+0000","last_peered":"2024-09-22T03:32:41.539057+0000","last_clean":"2024-09-22T03:32:41.539057+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:32:41.539057+0000","last_undegraded":"2024-09-22T03:32:41.539057+0000","last_fullsized":"2024-09-22T03:32:41.539057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808280+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.808280+0000","last_peered":"2024-09-22T03:32:13.808280+0000","last_clean":"2024-09-22T03:32:13.808280+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:13.808280+0000","last_undegraded":"2024-09-22T03:32:13.808280+0000","last_fullsized":"2024-09-22T03:32:13.808280+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"80'263","reported_seq":276,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.631509+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:32:40.631509+0000","last_peered":"2024-09-22T03:32:40.631509+0000","last_clean":"2024-09-22T03:32:40.631509+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:32:40.631509+0000","last_undegraded":"2024-09-22T03:32:40.631509+0000","last_fullsized":"2024-09-22T03:32:40.631509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814132+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:13.814132+0000","last_peered":"2024-09-22T03:32:13.814132+0000","last_clean":"2024-09-22T03:32:13.814132+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:13.814132+0000","last_undegraded":"2024-09-22T03:32:13.814132+0000","last_fullsized":"2024-09-22T03:32:13.814132+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"80'272","reported_seq":285,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.371025+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:32:41.371025+0000","last_peered":"2024-09-22T03:32:41.371025+0000","last_clean":"2024-09-22T03:32:41.371025+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:32:41.371025+0000","last_undegraded":"2024-09-22T03:32:41.371025+0000","last_fullsized":"2024-09-22T03:32:41.371025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2236416,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":2184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.820495+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:13.820495+0000","last_peered":"2024-09-22T03:32:13.820495+0000","last_clean":"2024-09-22T03:32:13.820495+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:13.820495+0000","last_undegraded":"2024-09-22T03:32:13.820495+0000","last_fullsized":"2024-09-22T03:32:13.820495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":35069952,"num_objects":536,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4281,"num_write_kb":34248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213984419840,"internally_reserved":0,"allocated":227074048,"data_stored":227074048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4265,"ondisk_log_size":4265,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213984419840,"internally_reserved":0,"allocated":227074048,"data_stored":227074048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":53132,"kb_used_data":53132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52244596,"statfs":{"total":53552873472,"available":53498466304,"internally_reserved":0,"allocated":54407168,"data_stored":54407168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":41948,"kb_used_data":41948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52255780,"statfs":{"total":53552873472,"available":53509918720,"internally_reserved":0,"allocated":42954752,"data_stored":42954752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":62364,"kb_used_data":62364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52235364,"statfs":{"total":53552873472,"available":53489012736,"internally_reserved":0,"allocated":63860736,"data_stored":63860736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":57844,"kb_used_data":57844,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52239884,"statfs":{"total":53552873472,"available":53493641216,"internally_reserved":0,"allocated":59232256,"data_stored":59232256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53492076544,"internally_reserved":0,"allocated":60796928,"data_stored":60796928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53486936064,"internally_reserved":0,"allocated":65937408,"data_stored":65937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53508272128,"internally_reserved":0,"allocated":44601344,"data_stored":44601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53492076544,"internally_reserved":0,"allocated":60796928,"data_stored":60796928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53486936064,"internally_reserved":0,"allocated":65937408,"data_stored":65937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53508272128,"internally_reserved":0,"allocated":44601344,"data_stored":44601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:32:46.540 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:32:46.541 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:46.541 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:32:46.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:46.674+0000 7f3fe6797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:46.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:46.683+0000 7f3fe6797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:46.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 4982 4966 1.17555 1.1875 0.0728202 0.106115 2024-09-22T03:32:46.976 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:46.997 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:32:46.998 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:32:47.019 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-22T03:32:46.043470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35660339,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4420,"num_write_kb":36144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4365,"ondisk_log_size":4365,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":215288,"kb_used_data":215288,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208975624,"statfs":{"total":214211493888,"available":213991038976,"internally_reserved":0,"allocated":220454912,"data_stored":220454912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000643"},"pg_stats":[{"pgid":"8.7","version":"80'335","reported_seq":348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.847472+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:32:41.847472+0000","last_peered":"2024-09-22T03:32:41.847472+0000","last_clean":"2024-09-22T03:32:41.847472+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:32:41.847472+0000","last_undegraded":"2024-09-22T03:32:41.847472+0000","last_fullsized":"2024-09-22T03:32:41.847472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.672573+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:32:41.672573+0000","last_peered":"2024-09-22T03:32:41.672573+0000","last_clean":"2024-09-22T03:32:41.672573+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:32:41.672573+0000","last_undegraded":"2024-09-22T03:32:41.672573+0000","last_fullsized":"2024-09-22T03:32:41.672573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.790364+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:32:41.790364+0000","last_peered":"2024-09-22T03:32:41.790364+0000","last_clean":"2024-09-22T03:32:41.790364+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:32:41.790364+0000","last_undegraded":"2024-09-22T03:32:41.790364+0000","last_fullsized":"2024-09-22T03:32:41.790364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"80'247","reported_seq":260,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.896832+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:32:41.896832+0000","last_peered":"2024-09-22T03:32:41.896832+0000","last_clean":"2024-09-22T03:32:41.896832+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:32:41.896832+0000","last_undegraded":"2024-09-22T03:32:41.896832+0000","last_fullsized":"2024-09-22T03:32:41.896832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"80'175","reported_seq":188,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.115878+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:32:41.115878+0000","last_peered":"2024-09-22T03:32:41.115878+0000","last_clean":"2024-09-22T03:32:41.115878+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:32:41.115878+0000","last_undegraded":"2024-09-22T03:32:41.115878+0000","last_fullsized":"2024-09-22T03:32:41.115878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"80'247","reported_seq":260,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.926663+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:32:40.926663+0000","last_peered":"2024-09-22T03:32:40.926663+0000","last_clean":"2024-09-22T03:32:40.926663+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:32:40.926663+0000","last_undegraded":"2024-09-22T03:32:40.926663+0000","last_fullsized":"2024-09-22T03:32:40.926663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"80'335","reported_seq":348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.310594+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:32:41.310594+0000","last_peered":"2024-09-22T03:32:41.310594+0000","last_clean":"2024-09-22T03:32:41.310594+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:32:41.310594+0000","last_undegraded":"2024-09-22T03:32:41.310594+0000","last_fullsized":"2024-09-22T03:32:41.310594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"80'271","reported_seq":284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.630635+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:32:41.630635+0000","last_peered":"2024-09-22T03:32:41.630635+0000","last_clean":"2024-09-22T03:32:41.630635+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:32:41.630635+0000","last_undegraded":"2024-09-22T03:32:41.630635+0000","last_fullsized":"2024-09-22T03:32:41.630635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.436981+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:32:41.436981+0000","last_peered":"2024-09-22T03:32:41.436981+0000","last_clean":"2024-09-22T03:32:41.436981+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:32:41.436981+0000","last_undegraded":"2024-09-22T03:32:41.436981+0000","last_fullsized":"2024-09-22T03:32:41.436981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":343,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.804356+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:13.804356+0000","last_peered":"2024-09-22T03:32:13.804356+0000","last_clean":"2024-09-22T03:32:13.804356+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:13.804356+0000","last_undegraded":"2024-09-22T03:32:13.804356+0000","last_fullsized":"2024-09-22T03:32:13.804356+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":209,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.818739+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:13.818739+0000","last_peered":"2024-09-22T03:32:13.818739+0000","last_clean":"2024-09-22T03:32:13.818739+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:13.818739+0000","last_undegraded":"2024-09-22T03:32:13.818739+0000","last_fullsized":"2024-09-22T03:32:13.818739+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"80'271","reported_seq":284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.942163+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:32:41.942163+0000","last_peered":"2024-09-22T03:32:41.942163+0000","last_clean":"2024-09-22T03:32:41.942163+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:32:41.942163+0000","last_undegraded":"2024-09-22T03:32:41.942163+0000","last_fullsized":"2024-09-22T03:32:41.942163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808550+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:13.808550+0000","last_peered":"2024-09-22T03:32:13.808550+0000","last_clean":"2024-09-22T03:32:13.808550+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:13.808550+0000","last_undegraded":"2024-09-22T03:32:13.808550+0000","last_fullsized":"2024-09-22T03:32:13.808550+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.743950+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:32:40.743950+0000","last_peered":"2024-09-22T03:32:40.743950+0000","last_clean":"2024-09-22T03:32:40.743950+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:32:40.743950+0000","last_undegraded":"2024-09-22T03:32:40.743950+0000","last_fullsized":"2024-09-22T03:32:40.743950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.816675+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.816675+0000","last_peered":"2024-09-22T03:32:13.816675+0000","last_clean":"2024-09-22T03:32:13.816675+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:13.816675+0000","last_undegraded":"2024-09-22T03:32:13.816675+0000","last_fullsized":"2024-09-22T03:32:13.816675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"80'223","reported_seq":236,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.832748+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:32:40.832748+0000","last_peered":"2024-09-22T03:32:40.832748+0000","last_clean":"2024-09-22T03:32:40.832748+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:32:40.832748+0000","last_undegraded":"2024-09-22T03:32:40.832748+0000","last_fullsized":"2024-09-22T03:32:40.832748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":210,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.805696+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:13.805696+0000","last_peered":"2024-09-22T03:32:13.805696+0000","last_clean":"2024-09-22T03:32:13.805696+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:13.805696+0000","last_undegraded":"2024-09-22T03:32:13.805696+0000","last_fullsized":"2024-09-22T03:32:13.805696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"80'303","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.724743+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:32:41.724743+0000","last_peered":"2024-09-22T03:32:41.724743+0000","last_clean":"2024-09-22T03:32:41.724743+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:32:41.724743+0000","last_undegraded":"2024-09-22T03:32:41.724743+0000","last_fullsized":"2024-09-22T03:32:41.724743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814199+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.814199+0000","last_peered":"2024-09-22T03:32:13.814199+0000","last_clean":"2024-09-22T03:32:13.814199+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:13.814199+0000","last_undegraded":"2024-09-22T03:32:13.814199+0000","last_fullsized":"2024-09-22T03:32:13.814199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"80'239","reported_seq":252,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.539057+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:32:41.539057+0000","last_peered":"2024-09-22T03:32:41.539057+0000","last_clean":"2024-09-22T03:32:41.539057+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:32:41.539057+0000","last_undegraded":"2024-09-22T03:32:41.539057+0000","last_fullsized":"2024-09-22T03:32:41.539057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":240,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.808280+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:13.808280+0000","last_peered":"2024-09-22T03:32:13.808280+0000","last_clean":"2024-09-22T03:32:13.808280+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:13.808280+0000","last_undegraded":"2024-09-22T03:32:13.808280+0000","last_fullsized":"2024-09-22T03:32:13.808280+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"80'263","reported_seq":276,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:40.631509+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:32:40.631509+0000","last_peered":"2024-09-22T03:32:40.631509+0000","last_clean":"2024-09-22T03:32:40.631509+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:32:40.631509+0000","last_undegraded":"2024-09-22T03:32:40.631509+0000","last_fullsized":"2024-09-22T03:32:40.631509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.814132+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:13.814132+0000","last_peered":"2024-09-22T03:32:13.814132+0000","last_clean":"2024-09-22T03:32:13.814132+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:13.814132+0000","last_undegraded":"2024-09-22T03:32:13.814132+0000","last_fullsized":"2024-09-22T03:32:13.814132+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"80'272","reported_seq":285,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:41.371025+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:32:41.371025+0000","last_peered":"2024-09-22T03:32:41.371025+0000","last_clean":"2024-09-22T03:32:41.371025+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:32:41.371025+0000","last_undegraded":"2024-09-22T03:32:41.371025+0000","last_fullsized":"2024-09-22T03:32:41.371025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2236416,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":2184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-22T03:32:13.820495+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:13.820495+0000","last_peered":"2024-09-22T03:32:13.820495+0000","last_clean":"2024-09-22T03:32:13.820495+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:13.820495+0000","last_undegraded":"2024-09-22T03:32:13.820495+0000","last_fullsized":"2024-09-22T03:32:13.820495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":35069952,"num_objects":536,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4281,"num_write_kb":34248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213984419840,"internally_reserved":0,"allocated":227074048,"data_stored":227074048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4265,"ondisk_log_size":4265,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213984419840,"internally_reserved":0,"allocated":227074048,"data_stored":227074048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":53132,"kb_used_data":53132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52244596,"statfs":{"total":53552873472,"available":53498466304,"internally_reserved":0,"allocated":54407168,"data_stored":54407168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":41948,"kb_used_data":41948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52255780,"statfs":{"total":53552873472,"available":53509918720,"internally_reserved":0,"allocated":42954752,"data_stored":42954752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":62364,"kb_used_data":62364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52235364,"statfs":{"total":53552873472,"available":53489012736,"internally_reserved":0,"allocated":63860736,"data_stored":63860736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":57844,"kb_used_data":57844,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52239884,"statfs":{"total":53552873472,"available":53493641216,"internally_reserved":0,"allocated":59232256,"data_stored":59232256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53492076544,"internally_reserved":0,"allocated":60796928,"data_stored":60796928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53486936064,"internally_reserved":0,"allocated":65937408,"data_stored":65937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53508272128,"internally_reserved":0,"allocated":44601344,"data_stored":44601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53492076544,"internally_reserved":0,"allocated":60796928,"data_stored":60796928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53486936064,"internally_reserved":0,"allocated":65937408,"data_stored":65937408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53508272128,"internally_reserved":0,"allocated":44601344,"data_stored":44601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53497135104,"internally_reserved":0,"allocated":55738368,"data_stored":55738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:32:47.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:47.096+0000 7fe5a40b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:47.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:47.101+0000 7fe5a40b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:47.864 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 5137 5121 1.17659 1.21094 0.134905 0.106124 2024-09-22T03:32:48.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 5311 5295 1.18181 1.35938 0.154213 0.105519 2024-09-22T03:32:49.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:49.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:49.384+0000 7f8e90ece640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:49.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:49.391+0000 7f8e90ece640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:49.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 5461 5445 1.18153 1.17188 0.0829192 0.105626 2024-09-22T03:32:50.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 5628 5612 1.18485 1.30469 0.0765195 0.105322 2024-09-22T03:32:51.577 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:51.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:51.690+0000 7ff794d03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:51.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:51.695+0000 7ff794d03640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:51.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 5770 5754 1.18287 1.10938 0.0417821 0.105408 2024-09-22T03:32:52.021 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:52.022 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:52.022 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:32:52.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:32:52.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.174+0000 7faecb197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.187+0000 7faecb197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:52.255 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:52.256 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:52.256 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:52.256 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:52.259 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:52.259 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:52.259 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:32:52.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.392+0000 7f31a9197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.404+0000 7f31a9197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.426 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:52.426 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:52.427 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:52.430 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:52.431 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:52.431 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:32:52.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.569+0000 7f44148b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.580+0000 7f44148b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.629 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:52.629 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:52.629 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:52.629 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:52.629 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:52.629 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:52.630 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:52.630 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:52.630 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:52.630 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:52.630 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:52.630 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:52.632 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:52.632 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:52.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:32:52.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.747+0000 7f21e1397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:52.756+0000 7f21e1397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:52.777 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:52.779 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:52.780 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:52.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 5909 5893 1.18038 1.08594 0.0570711 0.105703 2024-09-22T03:32:53.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:32:53.864945+0000 min lat: 0.0140532 max lat: 0.60614 avg lat: 0.105823 2024-09-22T03:32:53.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:32:53.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 6047 6031 1.17782 1.07812 0.059141 0.105823 2024-09-22T03:32:53.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:53.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:53.992+0000 7f195806c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:53.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:53.997+0000 7f195806c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:54.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 6189 6173 1.17615 1.10938 0.200841 0.106019 2024-09-22T03:32:55.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 6333 6317 1.17493 1.125 0.162016 0.10614 2024-09-22T03:32:56.110 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:32:56.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:56.252+0000 7f1c5406c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:56.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:56.260+0000 7f1c5406c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:56.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 16 6480 6464 1.17431 1.14844 0.127898 0.106353 2024-09-22T03:32:57.780 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:32:57.780 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:32:57.781 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:32:57.781 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:32:57.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 6627 6611 1.17372 1.14844 0.140175 0.106397 2024-09-22T03:32:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:57.930+0000 7fa599f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:57.941+0000 7fa599f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:58.029 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:58.030 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:58.030 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:32:58.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.144+0000 7f8367d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.154+0000 7f8367d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.172 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:58.172 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:58.172 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:58.172 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:58.172 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:58.172 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:58.172 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:58.173 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:58.173 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:58.173 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:58.173 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:58.173 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:58.176 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:58.176 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:58.176 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:32:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.282+0000 7f29db6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.290+0000 7f29db6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:58.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:58.314 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:58.314 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:32:58.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.445+0000 7fa927d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.459+0000 7fa927d17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.487 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:32:58.519 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:32:58.520 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:32:58.523 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:32:58.523 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:32:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.616+0000 7f20ec018640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:32:58.623+0000 7f20ec018640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:32:58.865 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 6752 6736 1.16934 0.976562 0.153966 0.106689 2024-09-22T03:32:59.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 6915 6899 1.1716 1.27344 0.0667501 0.106529 2024-09-22T03:33:00.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:00.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 16 7061 7045 1.17093 1.14062 0.14193 0.106577 2024-09-22T03:33:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:00.908+0000 7f260c917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:00.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:00.914+0000 7f260c917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:01.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 7207 7191 1.1703 1.14062 0.16126 0.106715 2024-09-22T03:33:02.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 7348 7332 1.16889 1.10156 0.160996 0.10678 2024-09-22T03:33:03.094 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:03.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:03.216+0000 7f4c5266c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:03.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:03.221+0000 7f4c5266c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:03.524 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:03.524 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:03.525 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:33:03.525 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:33:03.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:03.634+0000 7ff9b9997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:03.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:03.642+0000 7ff9b9997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:03.657 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:03.657 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:03.657 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:03.657 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:03.657 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:03.657 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:03.657 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:03.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:03.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:03.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:03.658 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:03.658 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:03.660 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:03.660 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:03.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:33:03.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:03.792+0000 7f7073e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:03.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:03.802+0000 7f7073e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:03.831 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:03.835 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:03.836 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:03.836 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:33:03.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 7493 7477 1.16817 1.13281 0.0753791 0.106865 2024-09-22T03:33:04.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:03.992+0000 7f9404d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:04.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:04.004+0000 7f9404d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:04.020 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:04.021 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:04.021 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:04.021 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:04.024 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:04.024 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:04.024 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:33:04.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:04.211+0000 7f48f3397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:04.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:04.225+0000 7f48f3397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:04.318 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:04.318 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:04.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 7630 7614 1.16625 1.07031 0.0467021 0.106963 2024-09-22T03:33:05.436 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:05.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:05.584+0000 7f64c2117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:05.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:05.589+0000 7f64c2117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:05.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 7765 7749 1.1641 1.05469 0.246906 0.107164 2024-09-22T03:33:06.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 7877 7861 1.15865 0.875 0.372237 0.107601 2024-09-22T03:33:07.726 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:07.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:07.843+0000 7f88a5357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:07.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:07.848+0000 7f88a5357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:07.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 8008 7992 1.15614 1.02344 0.15968 0.107956 2024-09-22T03:33:08.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 8146 8130 1.15472 1.07812 0.0607008 0.108091 2024-09-22T03:33:09.319 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:09.319 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:09.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:33:09.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:09.475+0000 7fb58aa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:09.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:09.484+0000 7fb58aa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:09.768 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:33:09.768 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:33:09.789 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-22T03:33:08.047546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":65200691,"num_objects":990,"num_object_clones":0,"num_object_copies":2970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8026,"num_write_kb":64992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7971,"ondisk_log_size":7971,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":397720,"kb_used_data":397720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208793192,"statfs":{"total":214211493888,"available":213804228608,"internally_reserved":0,"allocated":407265280,"data_stored":407265280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17252352,"num_objects":262,"num_object_clones":0,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2106,"num_write_kb":16848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002184"},"pg_stats":[{"pgid":"8.7","version":"84'567","reported_seq":592,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:05.350411+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:33:05.350411+0000","last_peered":"2024-09-22T03:33:05.350411+0000","last_clean":"2024-09-22T03:33:05.350411+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:33:05.350411+0000","last_undegraded":"2024-09-22T03:33:05.350411+0000","last_fullsized":"2024-09-22T03:33:05.350411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"84'519","reported_seq":544,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:05.999057+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:33:05.999057+0000","last_peered":"2024-09-22T03:33:05.999057+0000","last_clean":"2024-09-22T03:33:05.999057+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:33:05.999057+0000","last_undegraded":"2024-09-22T03:33:05.999057+0000","last_fullsized":"2024-09-22T03:33:05.999057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"84'479","reported_seq":504,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.419191+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:33:06.419191+0000","last_peered":"2024-09-22T03:33:06.419191+0000","last_clean":"2024-09-22T03:33:06.419191+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:33:06.419191+0000","last_undegraded":"2024-09-22T03:33:06.419191+0000","last_fullsized":"2024-09-22T03:33:06.419191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"84'495","reported_seq":520,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:03.887548+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:33:03.887548+0000","last_peered":"2024-09-22T03:33:03.887548+0000","last_clean":"2024-09-22T03:33:03.887548+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:33:03.887548+0000","last_undegraded":"2024-09-22T03:33:03.887548+0000","last_fullsized":"2024-09-22T03:33:03.887548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"84'319","reported_seq":344,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:05.929742+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:33:05.929742+0000","last_peered":"2024-09-22T03:33:05.929742+0000","last_clean":"2024-09-22T03:33:05.929742+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:33:05.929742+0000","last_undegraded":"2024-09-22T03:33:05.929742+0000","last_fullsized":"2024-09-22T03:33:05.929742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"84'535","reported_seq":560,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.377526+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:33:06.377526+0000","last_peered":"2024-09-22T03:33:06.377526+0000","last_clean":"2024-09-22T03:33:06.377526+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:33:06.377526+0000","last_undegraded":"2024-09-22T03:33:06.377526+0000","last_fullsized":"2024-09-22T03:33:06.377526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":536,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"84'495","reported_seq":520,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.450816+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:33:06.450816+0000","last_peered":"2024-09-22T03:33:06.450816+0000","last_clean":"2024-09-22T03:33:06.450816+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:33:06.450816+0000","last_undegraded":"2024-09-22T03:33:06.450816+0000","last_fullsized":"2024-09-22T03:33:06.450816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"84'471","reported_seq":496,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.271351+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:33:06.271351+0000","last_peered":"2024-09-22T03:33:06.271351+0000","last_clean":"2024-09-22T03:33:06.271351+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:33:06.271351+0000","last_undegraded":"2024-09-22T03:33:06.271351+0000","last_fullsized":"2024-09-22T03:33:06.271351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"84'455","reported_seq":480,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.780083+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:33:06.780083+0000","last_peered":"2024-09-22T03:33:06.780083+0000","last_clean":"2024-09-22T03:33:06.780083+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:33:06.780083+0000","last_undegraded":"2024-09-22T03:33:06.780083+0000","last_fullsized":"2024-09-22T03:33:06.780083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":355,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.765612+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:45.765612+0000","last_peered":"2024-09-22T03:32:45.765612+0000","last_clean":"2024-09-22T03:32:45.765612+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:45.765612+0000","last_undegraded":"2024-09-22T03:32:45.765612+0000","last_fullsized":"2024-09-22T03:32:45.765612+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":221,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.779889+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:45.779889+0000","last_peered":"2024-09-22T03:32:45.779889+0000","last_clean":"2024-09-22T03:32:45.779889+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:45.779889+0000","last_undegraded":"2024-09-22T03:32:45.779889+0000","last_fullsized":"2024-09-22T03:32:45.779889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"84'511","reported_seq":536,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:05.600474+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:33:05.600474+0000","last_peered":"2024-09-22T03:33:05.600474+0000","last_clean":"2024-09-22T03:33:05.600474+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:33:05.600474+0000","last_undegraded":"2024-09-22T03:33:05.600474+0000","last_fullsized":"2024-09-22T03:33:05.600474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.782955+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:45.782955+0000","last_peered":"2024-09-22T03:32:45.782955+0000","last_clean":"2024-09-22T03:32:45.782955+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:45.782955+0000","last_undegraded":"2024-09-22T03:32:45.782955+0000","last_fullsized":"2024-09-22T03:32:45.782955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"84'615","reported_seq":640,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.562823+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:33:06.562823+0000","last_peered":"2024-09-22T03:33:06.562823+0000","last_clean":"2024-09-22T03:33:06.562823+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:33:06.562823+0000","last_undegraded":"2024-09-22T03:33:06.562823+0000","last_fullsized":"2024-09-22T03:33:06.562823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.776397+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.776397+0000","last_peered":"2024-09-22T03:32:45.776397+0000","last_clean":"2024-09-22T03:32:45.776397+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:45.776397+0000","last_undegraded":"2024-09-22T03:32:45.776397+0000","last_fullsized":"2024-09-22T03:32:45.776397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"84'383","reported_seq":408,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:05.952852+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:33:05.952852+0000","last_peered":"2024-09-22T03:33:05.952852+0000","last_clean":"2024-09-22T03:33:05.952852+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:33:05.952852+0000","last_undegraded":"2024-09-22T03:33:05.952852+0000","last_fullsized":"2024-09-22T03:33:05.952852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":222,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.778581+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.778581+0000","last_peered":"2024-09-22T03:32:45.778581+0000","last_clean":"2024-09-22T03:32:45.778581+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:45.778581+0000","last_undegraded":"2024-09-22T03:32:45.778581+0000","last_fullsized":"2024-09-22T03:32:45.778581+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"84'495","reported_seq":520,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:05.466570+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:33:05.466570+0000","last_peered":"2024-09-22T03:33:05.466570+0000","last_clean":"2024-09-22T03:33:05.466570+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:33:05.466570+0000","last_undegraded":"2024-09-22T03:33:05.466570+0000","last_fullsized":"2024-09-22T03:33:05.466570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.772627+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.772627+0000","last_peered":"2024-09-22T03:32:45.772627+0000","last_clean":"2024-09-22T03:32:45.772627+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:45.772627+0000","last_undegraded":"2024-09-22T03:32:45.772627+0000","last_fullsized":"2024-09-22T03:32:45.772627+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"84'470","reported_seq":495,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.976981+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:33:06.976981+0000","last_peered":"2024-09-22T03:33:06.976981+0000","last_clean":"2024-09-22T03:33:06.976981+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:33:06.976981+0000","last_undegraded":"2024-09-22T03:33:06.976981+0000","last_fullsized":"2024-09-22T03:33:06.976981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3858432,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":471,"num_write_kb":3768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.769530+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.769530+0000","last_peered":"2024-09-22T03:32:45.769530+0000","last_clean":"2024-09-22T03:32:45.769530+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:45.769530+0000","last_undegraded":"2024-09-22T03:32:45.769530+0000","last_fullsized":"2024-09-22T03:32:45.769530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"84'495","reported_seq":520,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:04.997501+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:33:04.997501+0000","last_peered":"2024-09-22T03:33:04.997501+0000","last_clean":"2024-09-22T03:33:04.997501+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:33:04.997501+0000","last_undegraded":"2024-09-22T03:33:04.997501+0000","last_fullsized":"2024-09-22T03:33:04.997501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.771834+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:45.771834+0000","last_peered":"2024-09-22T03:32:45.771834+0000","last_clean":"2024-09-22T03:32:45.771834+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:45.771834+0000","last_undegraded":"2024-09-22T03:32:45.771834+0000","last_fullsized":"2024-09-22T03:32:45.771834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"84'567","reported_seq":592,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:06.904342+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:33:06.904342+0000","last_peered":"2024-09-22T03:33:06.904342+0000","last_clean":"2024-09-22T03:33:06.904342+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:33:06.904342+0000","last_undegraded":"2024-09-22T03:33:06.904342+0000","last_fullsized":"2024-09-22T03:33:06.904342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.781290+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:45.781290+0000","last_peered":"2024-09-22T03:32:45.781290+0000","last_clean":"2024-09-22T03:32:45.781290+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:45.781290+0000","last_undegraded":"2024-09-22T03:32:45.781290+0000","last_fullsized":"2024-09-22T03:32:45.781290+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":64610304,"num_objects":986,"num_object_clones":0,"num_object_copies":2958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7887,"num_write_kb":63096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213799796736,"internally_reserved":0,"allocated":411697152,"data_stored":411697152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7871,"ondisk_log_size":7871,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213799796736,"internally_reserved":0,"allocated":411697152,"data_stored":411697152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53453758464,"internally_reserved":0,"allocated":99115008,"data_stored":99115008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":95924,"kb_used_data":95924,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52201804,"statfs":{"total":53552873472,"available":53454647296,"internally_reserved":0,"allocated":98226176,"data_stored":98226176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":80312,"kb_used_data":80312,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52217416,"statfs":{"total":53552873472,"available":53470633984,"internally_reserved":0,"allocated":82239488,"data_stored":82239488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":114976,"kb_used_data":114976,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52182752,"statfs":{"total":53552873472,"available":53435138048,"internally_reserved":0,"allocated":117735424,"data_stored":117735424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":106508,"kb_used_data":106508,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52191220,"statfs":{"total":53552873472,"available":53443809280,"internally_reserved":0,"allocated":109064192,"data_stored":109064192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53453758464,"internally_reserved":0,"allocated":99115008,"data_stored":99115008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53442502656,"internally_reserved":0,"allocated":110370816,"data_stored":110370816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53433991168,"internally_reserved":0,"allocated":118882304,"data_stored":118882304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53469544448,"internally_reserved":0,"allocated":83329024,"data_stored":83329024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53453758464,"internally_reserved":0,"allocated":99115008,"data_stored":99115008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53442502656,"internally_reserved":0,"allocated":110370816,"data_stored":110370816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53433991168,"internally_reserved":0,"allocated":118882304,"data_stored":118882304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53469544448,"internally_reserved":0,"allocated":83329024,"data_stored":83329024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53453758464,"internally_reserved":0,"allocated":99115008,"data_stored":99115008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:33:09.866 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 8264 8248 1.15056 0.921875 0.100156 0.108563 2024-09-22T03:33:10.009 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:10.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:10.180+0000 7f1fdec6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:10.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:10.187+0000 7f1fdec6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:10.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 8400 8384 1.14902 1.0625 0.12939 0.108583 2024-09-22T03:33:11.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 8537 8521 1.14766 1.07031 0.0589766 0.108756 2024-09-22T03:33:12.468 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:12.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:12.618+0000 7f7d6142c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:12.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:12.624+0000 7f7d6142c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:12.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 8674 8658 1.14634 1.07031 0.134212 0.108901 2024-09-22T03:33:13.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:33:13.866849+0000 min lat: 0.0140532 max lat: 0.60614 avg lat: 0.108687 2024-09-22T03:33:13.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:33:13.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 8842 8826 1.14911 1.3125 0.0962533 0.108687 2024-09-22T03:33:14.791 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:14.791 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:14.792 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:33:14.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:33:14.795 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:14.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 8968 8952 1.14641 0.984375 0.126282 0.108837 2024-09-22T03:33:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:14.908+0000 7f8e38a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:14.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:14.918+0000 7f8e38a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:14.934 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:14.934 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:14.938 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:14.939 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:14.939 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:33:14.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:14.956+0000 7fa72de23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:14.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:14.962+0000 7fa72de23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:15.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:15.078+0000 7f7057557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:15.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:15.093+0000 7f7057557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:15.154 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:15.155 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:15.159 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:15.159 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:15.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:33:15.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:15.352+0000 7fa14af97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:15.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:15.365+0000 7fa14af97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:15.394 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:15.394 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:15.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:33:15.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:15.532+0000 7ffa11efa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:15.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:15.542+0000 7ffa11efa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:15.610 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:15.611 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:15.611 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:15.611 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:15.613 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:15.614 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:15.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 9105 9089 1.14518 1.07031 0.0719463 0.109036 2024-09-22T03:33:16.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 9248 9232 1.14473 1.11719 0.0803114 0.109104 2024-09-22T03:33:17.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:17.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:17.216+0000 7f198eefa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:17.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:17.224+0000 7f198eefa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:17.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 16 9397 9381 1.14503 1.16406 0.134924 0.109083 2024-09-22T03:33:18.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 9516 9500 1.14172 0.929688 0.24391 0.109351 2024-09-22T03:33:19.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:19.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:19.567+0000 7f0ecea6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:19.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:19.575+0000 7f0ecea6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:19.867 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 9648 9632 1.14004 1.03125 0.0548442 0.109517 2024-09-22T03:33:20.615 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:20.615 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:20.615 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:33:20.615 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:33:20.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:20.726+0000 7f686544f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:20.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:20.735+0000 7f686544f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:20.751 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:20.752 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:20.752 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:20.754 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:20.754 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:20.755 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:33:20.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 9778 9762 1.13819 1.01562 0.139561 0.10961 2024-09-22T03:33:20.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:20.872+0000 7f9ff0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:20.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:20.881+0000 7f9ff0597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:20.905 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:20.905 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:20.906 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:20.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:20.909 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:20.910 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:33:21.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:21.032+0000 7f8ca4cce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:21.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:21.043+0000 7f8ca4cce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:21.084 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:21.084 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:21.084 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:21.085 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:21.089 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:21.089 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:21.089 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:33:21.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:21.230+0000 7f5adfd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:21.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:21.245+0000 7f5adfd17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:21.300 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:21.301 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:21.301 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:21.301 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:21.303 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:21.304 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:21.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:21.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 9924 9908 1.13822 1.14062 0.106878 0.109697 2024-09-22T03:33:21.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:21.878+0000 7fc080c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:21.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:21.884+0000 7fc080c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:22.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 10065 10049 1.13769 1.10156 0.0567978 0.109743 2024-09-22T03:33:23.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 10208 10192 1.13739 1.11719 0.0751537 0.109803 2024-09-22T03:33:24.047 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:24.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:24.156+0000 7f18d7d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:24.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:24.161+0000 7f18d7d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:24.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 10343 10327 1.13623 1.05469 0.172648 0.109866 2024-09-22T03:33:25.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 10489 10473 1.13629 1.14062 0.0770136 0.109929 2024-09-22T03:33:26.304 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:26.304 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:26.305 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:33:26.305 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:26.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:26.429+0000 7f9cdd42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:26.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:26.434+0000 7f9cdd42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:26.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:26.462+0000 7feb52b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:26.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:26.470+0000 7feb52b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:26.814 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:33:26.815 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:33:26.836 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-22T03:33:26.051021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82010675,"num_objects":1247,"num_object_clones":0,"num_object_copies":3741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":10078,"num_write_kb":81408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10023,"ondisk_log_size":10023,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":498248,"kb_used_data":498248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208692664,"statfs":{"total":214211493888,"available":213701287936,"internally_reserved":0,"allocated":510205952,"data_stored":510205952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11378688,"num_objects":174,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002244"},"pg_stats":[{"pgid":"8.7","version":"84'655","reported_seq":680,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:19.641327+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:33:19.641327+0000","last_peered":"2024-09-22T03:33:19.641327+0000","last_clean":"2024-09-22T03:33:19.641327+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:33:19.641327+0000","last_undegraded":"2024-09-22T03:33:19.641327+0000","last_fullsized":"2024-09-22T03:33:19.641327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"84'647","reported_seq":672,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.773515+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:33:21.773515+0000","last_peered":"2024-09-22T03:33:21.773515+0000","last_clean":"2024-09-22T03:33:21.773515+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:33:21.773515+0000","last_undegraded":"2024-09-22T03:33:21.773515+0000","last_fullsized":"2024-09-22T03:33:21.773515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"84'567","reported_seq":592,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.895298+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:33:21.895298+0000","last_peered":"2024-09-22T03:33:21.895298+0000","last_clean":"2024-09-22T03:33:21.895298+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:33:21.895298+0000","last_undegraded":"2024-09-22T03:33:21.895298+0000","last_fullsized":"2024-09-22T03:33:21.895298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"84'599","reported_seq":624,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.425835+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:33:21.425835+0000","last_peered":"2024-09-22T03:33:21.425835+0000","last_clean":"2024-09-22T03:33:21.425835+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:33:21.425835+0000","last_undegraded":"2024-09-22T03:33:21.425835+0000","last_fullsized":"2024-09-22T03:33:21.425835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"84'487","reported_seq":512,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.471480+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:33:21.471480+0000","last_peered":"2024-09-22T03:33:21.471480+0000","last_clean":"2024-09-22T03:33:21.471480+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:33:21.471480+0000","last_undegraded":"2024-09-22T03:33:21.471480+0000","last_fullsized":"2024-09-22T03:33:21.471480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"84'647","reported_seq":672,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.130930+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:33:21.130930+0000","last_peered":"2024-09-22T03:33:21.130930+0000","last_clean":"2024-09-22T03:33:21.130930+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:33:21.130930+0000","last_undegraded":"2024-09-22T03:33:21.130930+0000","last_fullsized":"2024-09-22T03:33:21.130930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"84'623","reported_seq":648,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:19.878888+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:33:19.878888+0000","last_peered":"2024-09-22T03:33:19.878888+0000","last_clean":"2024-09-22T03:33:19.878888+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:33:19.878888+0000","last_undegraded":"2024-09-22T03:33:19.878888+0000","last_fullsized":"2024-09-22T03:33:19.878888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"84'591","reported_seq":616,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.658593+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:33:21.658593+0000","last_peered":"2024-09-22T03:33:21.658593+0000","last_clean":"2024-09-22T03:33:21.658593+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:33:21.658593+0000","last_undegraded":"2024-09-22T03:33:21.658593+0000","last_fullsized":"2024-09-22T03:33:21.658593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"84'607","reported_seq":632,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.722183+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:33:21.722183+0000","last_peered":"2024-09-22T03:33:21.722183+0000","last_clean":"2024-09-22T03:33:21.722183+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:33:21.722183+0000","last_undegraded":"2024-09-22T03:33:21.722183+0000","last_fullsized":"2024-09-22T03:33:21.722183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":355,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.765612+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:45.765612+0000","last_peered":"2024-09-22T03:32:45.765612+0000","last_clean":"2024-09-22T03:32:45.765612+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:45.765612+0000","last_undegraded":"2024-09-22T03:32:45.765612+0000","last_fullsized":"2024-09-22T03:32:45.765612+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":221,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.779889+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:45.779889+0000","last_peered":"2024-09-22T03:32:45.779889+0000","last_clean":"2024-09-22T03:32:45.779889+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:45.779889+0000","last_undegraded":"2024-09-22T03:32:45.779889+0000","last_fullsized":"2024-09-22T03:32:45.779889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"84'631","reported_seq":656,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:20.266884+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:33:20.266884+0000","last_peered":"2024-09-22T03:33:20.266884+0000","last_clean":"2024-09-22T03:33:20.266884+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:33:20.266884+0000","last_undegraded":"2024-09-22T03:33:20.266884+0000","last_fullsized":"2024-09-22T03:33:20.266884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.782955+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:45.782955+0000","last_peered":"2024-09-22T03:32:45.782955+0000","last_clean":"2024-09-22T03:32:45.782955+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:45.782955+0000","last_undegraded":"2024-09-22T03:32:45.782955+0000","last_fullsized":"2024-09-22T03:32:45.782955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"84'719","reported_seq":744,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.649139+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:33:21.649139+0000","last_peered":"2024-09-22T03:33:21.649139+0000","last_clean":"2024-09-22T03:33:21.649139+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:33:21.649139+0000","last_undegraded":"2024-09-22T03:33:21.649139+0000","last_fullsized":"2024-09-22T03:33:21.649139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.776397+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.776397+0000","last_peered":"2024-09-22T03:32:45.776397+0000","last_clean":"2024-09-22T03:32:45.776397+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:45.776397+0000","last_undegraded":"2024-09-22T03:32:45.776397+0000","last_fullsized":"2024-09-22T03:32:45.776397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"84'519","reported_seq":544,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.525257+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:33:21.525257+0000","last_peered":"2024-09-22T03:33:21.525257+0000","last_clean":"2024-09-22T03:33:21.525257+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:33:21.525257+0000","last_undegraded":"2024-09-22T03:33:21.525257+0000","last_fullsized":"2024-09-22T03:33:21.525257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":222,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.778581+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.778581+0000","last_peered":"2024-09-22T03:32:45.778581+0000","last_clean":"2024-09-22T03:32:45.778581+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:45.778581+0000","last_undegraded":"2024-09-22T03:32:45.778581+0000","last_fullsized":"2024-09-22T03:32:45.778581+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"84'591","reported_seq":616,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.208542+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:33:21.208542+0000","last_peered":"2024-09-22T03:33:21.208542+0000","last_clean":"2024-09-22T03:33:21.208542+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:33:21.208542+0000","last_undegraded":"2024-09-22T03:33:21.208542+0000","last_fullsized":"2024-09-22T03:33:21.208542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.772627+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.772627+0000","last_peered":"2024-09-22T03:32:45.772627+0000","last_clean":"2024-09-22T03:32:45.772627+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:45.772627+0000","last_undegraded":"2024-09-22T03:32:45.772627+0000","last_fullsized":"2024-09-22T03:32:45.772627+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"84'655","reported_seq":680,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.950822+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:33:21.950822+0000","last_peered":"2024-09-22T03:33:21.950822+0000","last_clean":"2024-09-22T03:33:21.950822+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:33:21.950822+0000","last_undegraded":"2024-09-22T03:33:21.950822+0000","last_fullsized":"2024-09-22T03:33:21.950822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.769530+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.769530+0000","last_peered":"2024-09-22T03:32:45.769530+0000","last_clean":"2024-09-22T03:32:45.769530+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:45.769530+0000","last_undegraded":"2024-09-22T03:32:45.769530+0000","last_fullsized":"2024-09-22T03:32:45.769530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"84'639","reported_seq":664,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.558183+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:33:21.558183+0000","last_peered":"2024-09-22T03:33:21.558183+0000","last_clean":"2024-09-22T03:33:21.558183+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:33:21.558183+0000","last_undegraded":"2024-09-22T03:33:21.558183+0000","last_fullsized":"2024-09-22T03:33:21.558183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.771834+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:45.771834+0000","last_peered":"2024-09-22T03:32:45.771834+0000","last_clean":"2024-09-22T03:32:45.771834+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:45.771834+0000","last_undegraded":"2024-09-22T03:32:45.771834+0000","last_fullsized":"2024-09-22T03:32:45.771834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"84'746","reported_seq":771,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:21.973614+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:33:21.973614+0000","last_peered":"2024-09-22T03:33:21.973614+0000","last_clean":"2024-09-22T03:33:21.973614+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:33:21.973614+0000","last_undegraded":"2024-09-22T03:33:21.973614+0000","last_fullsized":"2024-09-22T03:33:21.973614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6119424,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.781290+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:45.781290+0000","last_peered":"2024-09-22T03:32:45.781290+0000","last_clean":"2024-09-22T03:32:45.781290+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:45.781290+0000","last_undegraded":"2024-09-22T03:32:45.781290+0000","last_fullsized":"2024-09-22T03:32:45.781290+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":81420288,"num_objects":1243,"num_object_clones":0,"num_object_copies":3729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9939,"num_write_kb":79512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213694459904,"internally_reserved":0,"allocated":517033984,"data_stored":517033984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9923,"ondisk_log_size":9923,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213694541824,"internally_reserved":0,"allocated":516952064,"data_stored":516952064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53427163136,"internally_reserved":0,"allocated":125710336,"data_stored":125710336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706283,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":121208,"kb_used_data":121208,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52176520,"statfs":{"total":53552873472,"available":53428756480,"internally_reserved":0,"allocated":124116992,"data_stored":124116992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706283,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":99512,"kb_used_data":99512,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52198216,"statfs":{"total":53552873472,"available":53450973184,"internally_reserved":0,"allocated":101900288,"data_stored":101900288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706283,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":144056,"kb_used_data":144056,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52153672,"statfs":{"total":53552873472,"available":53405360128,"internally_reserved":0,"allocated":147513344,"data_stored":147513344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706283,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":133472,"kb_used_data":133472,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52164256,"statfs":{"total":53552873472,"available":53416198144,"internally_reserved":0,"allocated":136675328,"data_stored":136675328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53427163136,"internally_reserved":0,"allocated":125710336,"data_stored":125710336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53414330368,"internally_reserved":0,"allocated":138543104,"data_stored":138543104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53403394048,"internally_reserved":0,"allocated":149479424,"data_stored":149479424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53449654272,"internally_reserved":0,"allocated":103219200,"data_stored":103219200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53427163136,"internally_reserved":0,"allocated":125710336,"data_stored":125710336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53414330368,"internally_reserved":0,"allocated":138543104,"data_stored":138543104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53403312128,"internally_reserved":0,"allocated":149561344,"data_stored":149561344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53449654272,"internally_reserved":0,"allocated":103219200,"data_stored":103219200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53427163136,"internally_reserved":0,"allocated":125710336,"data_stored":125710336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:33:26.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 10597 10581 1.13228 0.84375 0.0616338 0.110081 2024-09-22T03:33:27.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 10725 10709 1.13049 1 0.27956 0.110454 2024-09-22T03:33:28.637 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:28.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:28.781+0000 7fa808e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:28.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:28.789+0000 7fa808e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:28.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 10840 10824 1.12739 0.898438 0.123275 0.110745 2024-09-22T03:33:29.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 10971 10955 1.12603 1.02344 0.129456 0.110892 2024-09-22T03:33:30.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 11079 11063 1.12236 0.84375 0.144849 0.111161 2024-09-22T03:33:30.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:31.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:31.085+0000 7f5bb8258640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:31.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:31.091+0000 7f5bb8258640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:31.838 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:31.838 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:31.838 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:33:31.839 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:33:31.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 16 11221 11205 1.12219 1.10938 0.120335 0.111324 2024-09-22T03:33:32.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:31.993+0000 7f8319997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:32.008+0000 7f8319997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.135 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:32.135 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:32.135 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:32.135 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:32.135 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:32.135 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:32.136 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:32.136 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:32.136 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:32.136 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:32.136 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:32.136 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:32.140 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:32.140 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:32.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:33:32.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:32.319+0000 7fbc1f917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:32.333+0000 7fbc1f917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:32.397 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:32.398 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:32.398 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:32.400 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:32.400 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:32.400 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:33:32.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:32.539+0000 7fe3c262c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:32.549+0000 7fe3c262c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.563 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:32.564 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:32.567 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:32.567 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:32.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:33:32.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:32.688+0000 7fb10deb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:32.701+0000 7fb10deb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:32.751 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:32.752 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:32.753 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:32.753 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:32.753 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:32.755 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:32.756 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:32.868 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 11362 11346 1.12193 1.10156 0.15174 0.111284 2024-09-22T03:33:33.300 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:33.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:33.412+0000 7fe6bcc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:33.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:33.417+0000 7fe6bcc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:33.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:33:33.868630+0000 min lat: 0.0140532 max lat: 0.60614 avg lat: 0.111537 2024-09-22T03:33:33.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:33:33.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 11485 11469 1.11992 0.960938 0.102993 0.111537 2024-09-22T03:33:34.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 11605 11589 1.11766 0.9375 0.0760066 0.111696 2024-09-22T03:33:35.584 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:35.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:35.700+0000 7f9d35a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:35.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:35.706+0000 7f9d35a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:35.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 11729 11713 1.11585 0.96875 0.384508 0.111951 2024-09-22T03:33:36.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 11851 11835 1.11388 0.953125 0.157239 0.112093 2024-09-22T03:33:37.757 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:37.757 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:37.757 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:33:37.757 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:33:37.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 11981 11965 1.11271 1.01562 0.108076 0.11212 2024-09-22T03:33:37.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:37.882+0000 7f9a66397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:37.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:37.891+0000 7f9a66397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:37.908 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:37.911 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:37.911 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:37.911 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:33:37.941 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:38.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.032+0000 7f9f31b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.043+0000 7f9f31b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.066 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:38.067 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:38.071 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:38.071 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:38.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:33:38.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.075+0000 7f4b7c0ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.081+0000 7f4b7c0ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.216+0000 7f9c4b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.225+0000 7f9c4b997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.243 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:38.243 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:38.243 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:38.243 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:38.243 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:38.243 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:38.244 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:38.244 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:38.244 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:38.244 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:38.244 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:38.244 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:38.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:38.247 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:38.247 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:33:38.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.447+0000 7f13a7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:38.457+0000 7f13a7997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:38.504 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:38.505 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:38.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:38.509 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:38.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 12102 12086 1.11074 0.945312 0.131157 0.112438 2024-09-22T03:33:39.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 12243 12227 1.11064 1.10156 0.0595848 0.112427 2024-09-22T03:33:40.345 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:40.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:40.456+0000 7f3a03117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:40.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:40.462+0000 7f3a03117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:40.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 12383 12367 1.11044 1.09375 0.13104 0.112437 2024-09-22T03:33:41.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 12521 12505 1.11007 1.07812 0.136949 0.112502 2024-09-22T03:33:42.733 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:42.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:42.851+0000 7f35da66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:42.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:42.856+0000 7f35da66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:42.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 12648 12632 1.10875 0.992188 0.244542 0.112568 2024-09-22T03:33:43.510 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:43.510 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:43.510 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:33:43.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:43.640+0000 7fa9d810e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:43.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:43.651+0000 7fa9d810e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:43.869 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 15 12759 12744 1.10615 0.875 0.0402047 0.112925 2024-09-22T03:33:43.970 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:33:43.970 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:33:43.985 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-22T03:33:42.054050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103293491,"num_objects":1572,"num_object_clones":0,"num_object_copies":4716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":12676,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12621,"ondisk_log_size":12621,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":628120,"kb_used_data":628120,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208562792,"statfs":{"total":214211493888,"available":213568299008,"internally_reserved":0,"allocated":643194880,"data_stored":643194880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15753216,"num_objects":241,"num_object_clones":0,"num_object_copies":723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1923,"num_write_kb":15384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002264"},"pg_stats":[{"pgid":"8.7","version":"84'823","reported_seq":848,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:40.109771+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:33:40.109771+0000","last_peered":"2024-09-22T03:33:40.109771+0000","last_clean":"2024-09-22T03:33:40.109771+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:33:40.109771+0000","last_undegraded":"2024-09-22T03:33:40.109771+0000","last_fullsized":"2024-09-22T03:33:40.109771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":103,"num_object_clones":0,"num_object_copies":309,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"84'783","reported_seq":808,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:40.713185+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:33:40.713185+0000","last_peered":"2024-09-22T03:33:40.713185+0000","last_clean":"2024-09-22T03:33:40.713185+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:33:40.713185+0000","last_undegraded":"2024-09-22T03:33:40.713185+0000","last_fullsized":"2024-09-22T03:33:40.713185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"84'743","reported_seq":768,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.288751+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:33:41.288751+0000","last_peered":"2024-09-22T03:33:41.288751+0000","last_clean":"2024-09-22T03:33:41.288751+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:33:41.288751+0000","last_undegraded":"2024-09-22T03:33:41.288751+0000","last_fullsized":"2024-09-22T03:33:41.288751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"84'695","reported_seq":720,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:36.545224+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:33:36.545224+0000","last_peered":"2024-09-22T03:33:36.545224+0000","last_clean":"2024-09-22T03:33:36.545224+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:33:36.545224+0000","last_undegraded":"2024-09-22T03:33:36.545224+0000","last_fullsized":"2024-09-22T03:33:36.545224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":696,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"84'655","reported_seq":680,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.508267+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:33:41.508267+0000","last_peered":"2024-09-22T03:33:41.508267+0000","last_clean":"2024-09-22T03:33:41.508267+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:33:41.508267+0000","last_undegraded":"2024-09-22T03:33:41.508267+0000","last_fullsized":"2024-09-22T03:33:41.508267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"84'815","reported_seq":840,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:39.895445+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:33:39.895445+0000","last_peered":"2024-09-22T03:33:39.895445+0000","last_clean":"2024-09-22T03:33:39.895445+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:33:39.895445+0000","last_undegraded":"2024-09-22T03:33:39.895445+0000","last_fullsized":"2024-09-22T03:33:39.895445+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"84'831","reported_seq":856,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:40.865695+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:33:40.865695+0000","last_peered":"2024-09-22T03:33:40.865695+0000","last_clean":"2024-09-22T03:33:40.865695+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:33:40.865695+0000","last_undegraded":"2024-09-22T03:33:40.865695+0000","last_fullsized":"2024-09-22T03:33:40.865695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"84'799","reported_seq":824,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.164053+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:33:41.164053+0000","last_peered":"2024-09-22T03:33:41.164053+0000","last_clean":"2024-09-22T03:33:41.164053+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:33:41.164053+0000","last_undegraded":"2024-09-22T03:33:41.164053+0000","last_fullsized":"2024-09-22T03:33:41.164053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"84'751","reported_seq":776,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.979361+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:33:41.979361+0000","last_peered":"2024-09-22T03:33:41.979361+0000","last_clean":"2024-09-22T03:33:41.979361+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:33:41.979361+0000","last_undegraded":"2024-09-22T03:33:41.979361+0000","last_fullsized":"2024-09-22T03:33:41.979361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":355,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.765612+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:45.765612+0000","last_peered":"2024-09-22T03:32:45.765612+0000","last_clean":"2024-09-22T03:32:45.765612+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:45.765612+0000","last_undegraded":"2024-09-22T03:32:45.765612+0000","last_fullsized":"2024-09-22T03:32:45.765612+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":221,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.779889+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:45.779889+0000","last_peered":"2024-09-22T03:32:45.779889+0000","last_clean":"2024-09-22T03:32:45.779889+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:45.779889+0000","last_undegraded":"2024-09-22T03:32:45.779889+0000","last_fullsized":"2024-09-22T03:32:45.779889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"84'807","reported_seq":832,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.365654+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:33:41.365654+0000","last_peered":"2024-09-22T03:33:41.365654+0000","last_clean":"2024-09-22T03:33:41.365654+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:33:41.365654+0000","last_undegraded":"2024-09-22T03:33:41.365654+0000","last_fullsized":"2024-09-22T03:33:41.365654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.782955+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:45.782955+0000","last_peered":"2024-09-22T03:32:45.782955+0000","last_clean":"2024-09-22T03:32:45.782955+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:45.782955+0000","last_undegraded":"2024-09-22T03:32:45.782955+0000","last_fullsized":"2024-09-22T03:32:45.782955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"84'911","reported_seq":936,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.779758+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:33:41.779758+0000","last_peered":"2024-09-22T03:33:41.779758+0000","last_clean":"2024-09-22T03:33:41.779758+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:33:41.779758+0000","last_undegraded":"2024-09-22T03:33:41.779758+0000","last_fullsized":"2024-09-22T03:33:41.779758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.776397+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.776397+0000","last_peered":"2024-09-22T03:32:45.776397+0000","last_clean":"2024-09-22T03:32:45.776397+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:45.776397+0000","last_undegraded":"2024-09-22T03:32:45.776397+0000","last_fullsized":"2024-09-22T03:32:45.776397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"84'671","reported_seq":696,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.350357+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:33:41.350357+0000","last_peered":"2024-09-22T03:33:41.350357+0000","last_clean":"2024-09-22T03:33:41.350357+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:33:41.350357+0000","last_undegraded":"2024-09-22T03:33:41.350357+0000","last_fullsized":"2024-09-22T03:33:41.350357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":5376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":222,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.778581+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.778581+0000","last_peered":"2024-09-22T03:32:45.778581+0000","last_clean":"2024-09-22T03:32:45.778581+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:45.778581+0000","last_undegraded":"2024-09-22T03:32:45.778581+0000","last_fullsized":"2024-09-22T03:32:45.778581+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"84'680","reported_seq":705,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.686680+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:33:41.686680+0000","last_peered":"2024-09-22T03:33:41.686680+0000","last_clean":"2024-09-22T03:33:41.686680+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:33:41.686680+0000","last_undegraded":"2024-09-22T03:33:41.686680+0000","last_fullsized":"2024-09-22T03:33:41.686680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5578752,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":5448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.772627+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.772627+0000","last_peered":"2024-09-22T03:32:45.772627+0000","last_clean":"2024-09-22T03:32:45.772627+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:45.772627+0000","last_undegraded":"2024-09-22T03:32:45.772627+0000","last_fullsized":"2024-09-22T03:32:45.772627+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"84'863","reported_seq":888,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.964581+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:33:41.964581+0000","last_peered":"2024-09-22T03:33:41.964581+0000","last_clean":"2024-09-22T03:33:41.964581+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:33:41.964581+0000","last_undegraded":"2024-09-22T03:33:41.964581+0000","last_fullsized":"2024-09-22T03:33:41.964581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.769530+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.769530+0000","last_peered":"2024-09-22T03:32:45.769530+0000","last_clean":"2024-09-22T03:32:45.769530+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:45.769530+0000","last_undegraded":"2024-09-22T03:32:45.769530+0000","last_fullsized":"2024-09-22T03:32:45.769530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"84'799","reported_seq":824,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:41.463172+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:33:41.463172+0000","last_peered":"2024-09-22T03:33:41.463172+0000","last_clean":"2024-09-22T03:33:41.463172+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:33:41.463172+0000","last_undegraded":"2024-09-22T03:33:41.463172+0000","last_fullsized":"2024-09-22T03:33:41.463172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.771834+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:45.771834+0000","last_peered":"2024-09-22T03:32:45.771834+0000","last_clean":"2024-09-22T03:32:45.771834+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:45.771834+0000","last_undegraded":"2024-09-22T03:32:45.771834+0000","last_fullsized":"2024-09-22T03:32:45.771834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"84'895","reported_seq":920,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:39.748944+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:33:39.748944+0000","last_peered":"2024-09-22T03:33:39.748944+0000","last_clean":"2024-09-22T03:33:39.748944+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:33:39.748944+0000","last_undegraded":"2024-09-22T03:33:39.748944+0000","last_fullsized":"2024-09-22T03:33:39.748944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.781290+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:45.781290+0000","last_peered":"2024-09-22T03:32:45.781290+0000","last_clean":"2024-09-22T03:32:45.781290+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:45.781290+0000","last_undegraded":"2024-09-22T03:32:45.781290+0000","last_fullsized":"2024-09-22T03:32:45.781290+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":102703104,"num_objects":1568,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":100296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213562736640,"internally_reserved":0,"allocated":648757248,"data_stored":648757248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12521,"ondisk_log_size":12521,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213562736640,"internally_reserved":0,"allocated":648757248,"data_stored":648757248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53394161664,"internally_reserved":0,"allocated":158711808,"data_stored":158711808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706303,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":153580,"kb_used_data":153580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52144148,"statfs":{"total":53552873472,"available":53395607552,"internally_reserved":0,"allocated":157265920,"data_stored":157265920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706303,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":124368,"kb_used_data":124368,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52173360,"statfs":{"total":53552873472,"available":53425520640,"internally_reserved":0,"allocated":127352832,"data_stored":127352832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706303,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":182228,"kb_used_data":182228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52115500,"statfs":{"total":53552873472,"available":53366272000,"internally_reserved":0,"allocated":186601472,"data_stored":186601472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706303,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":167944,"kb_used_data":167944,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52129784,"statfs":{"total":53552873472,"available":53380898816,"internally_reserved":0,"allocated":171974656,"data_stored":171974656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53394161664,"internally_reserved":0,"allocated":158711808,"data_stored":158711808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53379579904,"internally_reserved":0,"allocated":173293568,"data_stored":173293568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53364633600,"internally_reserved":0,"allocated":188239872,"data_stored":188239872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53424361472,"internally_reserved":0,"allocated":128512000,"data_stored":128512000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53394161664,"internally_reserved":0,"allocated":158711808,"data_stored":158711808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53379579904,"internally_reserved":0,"allocated":173293568,"data_stored":173293568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53364633600,"internally_reserved":0,"allocated":188239872,"data_stored":188239872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53424361472,"internally_reserved":0,"allocated":128512000,"data_stored":128512000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53394161664,"internally_reserved":0,"allocated":158711808,"data_stored":158711808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.1183 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 12759 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.1061 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.122208 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.40625 2024-09-22T03:33:44.870 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 0.84375 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 141 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 15.6426 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 180 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 108 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.112934 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.0616696 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.60614 2024-09-22T03:33:44.871 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0140532 2024-09-22T03:33:44.902 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:33:45.056 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:45.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:45.226+0000 7f263b86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:45.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:45.236+0000 7f263b86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:47.474 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:47.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:47.629+0000 7fb3ff0fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:47.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:47.635+0000 7fb3ff0fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:48.987 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:48.988 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:48.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:33:49.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:49.130+0000 7f7726ca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:49.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:49.139+0000 7f7726ca9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:33:49.431 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:33:49.447 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-22T03:33:48.055216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82698827,"num_objects":1258,"num_object_clones":0,"num_object_copies":3774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13241,"num_write_kb":103969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13186,"ondisk_log_size":13186,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":567320,"kb_used_data":567320,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208623592,"statfs":{"total":214211493888,"available":213630558208,"internally_reserved":0,"allocated":580935680,"data_stored":580935680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9912296,"num_objects":-151,"num_object_clones":0,"num_object_copies":-453,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1869,"num_write_kb":12209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002367"},"pg_stats":[{"pgid":"8.7","version":"84'872","reported_seq":897,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.777894+0000","last_change":"2024-09-22T03:32:12.887003+0000","last_active":"2024-09-22T03:33:46.777894+0000","last_peered":"2024-09-22T03:33:46.777894+0000","last_clean":"2024-09-22T03:33:46.777894+0000","last_became_active":"2024-09-22T03:32:12.882824+0000","last_became_peered":"2024-09-22T03:32:12.882824+0000","last_unstale":"2024-09-22T03:33:46.777894+0000","last_undegraded":"2024-09-22T03:33:46.777894+0000","last_fullsized":"2024-09-22T03:33:46.777894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":873,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"84'835","reported_seq":860,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.632339+0000","last_change":"2024-09-22T03:32:13.007377+0000","last_active":"2024-09-22T03:33:46.632339+0000","last_peered":"2024-09-22T03:33:46.632339+0000","last_clean":"2024-09-22T03:33:46.632339+0000","last_became_active":"2024-09-22T03:32:13.002569+0000","last_became_peered":"2024-09-22T03:32:13.002569+0000","last_unstale":"2024-09-22T03:33:46.632339+0000","last_undegraded":"2024-09-22T03:33:46.632339+0000","last_fullsized":"2024-09-22T03:33:46.632339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4784128,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":836,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"84'773","reported_seq":798,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.784743+0000","last_change":"2024-09-22T03:32:12.934145+0000","last_active":"2024-09-22T03:33:46.784743+0000","last_peered":"2024-09-22T03:33:46.784743+0000","last_clean":"2024-09-22T03:33:46.784743+0000","last_became_active":"2024-09-22T03:32:12.903614+0000","last_became_peered":"2024-09-22T03:32:12.903614+0000","last_unstale":"2024-09-22T03:33:46.784743+0000","last_undegraded":"2024-09-22T03:33:46.784743+0000","last_fullsized":"2024-09-22T03:33:46.784743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":774,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"84'734","reported_seq":759,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.920871+0000","last_change":"2024-09-22T03:32:12.953169+0000","last_active":"2024-09-22T03:33:46.920871+0000","last_peered":"2024-09-22T03:33:46.920871+0000","last_clean":"2024-09-22T03:33:46.920871+0000","last_became_active":"2024-09-22T03:32:12.940292+0000","last_became_peered":"2024-09-22T03:32:12.940292+0000","last_unstale":"2024-09-22T03:33:46.920871+0000","last_undegraded":"2024-09-22T03:33:46.920871+0000","last_fullsized":"2024-09-22T03:33:46.920871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":735,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3","version":"84'675","reported_seq":700,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.676244+0000","last_change":"2024-09-22T03:32:12.898969+0000","last_active":"2024-09-22T03:33:46.676244+0000","last_peered":"2024-09-22T03:33:46.676244+0000","last_clean":"2024-09-22T03:33:46.676244+0000","last_became_active":"2024-09-22T03:32:12.872364+0000","last_became_peered":"2024-09-22T03:32:12.872364+0000","last_unstale":"2024-09-22T03:33:46.676244+0000","last_undegraded":"2024-09-22T03:33:46.676244+0000","last_fullsized":"2024-09-22T03:33:46.676244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"84'837","reported_seq":862,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.961223+0000","last_change":"2024-09-22T03:32:12.939562+0000","last_active":"2024-09-22T03:33:46.961223+0000","last_peered":"2024-09-22T03:33:46.961223+0000","last_clean":"2024-09-22T03:33:46.961223+0000","last_became_active":"2024-09-22T03:32:12.935381+0000","last_became_peered":"2024-09-22T03:32:12.935381+0000","last_unstale":"2024-09-22T03:33:46.961223+0000","last_undegraded":"2024-09-22T03:33:46.961223+0000","last_fullsized":"2024-09-22T03:33:46.961223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":837,"log_dups_size":0,"ondisk_log_size":837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":838,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"84'889","reported_seq":914,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.767634+0000","last_change":"2024-09-22T03:32:12.900014+0000","last_active":"2024-09-22T03:33:46.767634+0000","last_peered":"2024-09-22T03:33:46.767634+0000","last_clean":"2024-09-22T03:33:46.767634+0000","last_became_active":"2024-09-22T03:32:12.890479+0000","last_became_peered":"2024-09-22T03:32:12.890479+0000","last_unstale":"2024-09-22T03:33:46.767634+0000","last_undegraded":"2024-09-22T03:33:46.767634+0000","last_fullsized":"2024-09-22T03:33:46.767634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":890,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"84'828","reported_seq":853,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.773624+0000","last_change":"2024-09-22T03:32:12.987237+0000","last_active":"2024-09-22T03:33:46.773624+0000","last_peered":"2024-09-22T03:33:46.773624+0000","last_clean":"2024-09-22T03:33:46.773624+0000","last_became_active":"2024-09-22T03:32:12.983261+0000","last_became_peered":"2024-09-22T03:32:12.983261+0000","last_unstale":"2024-09-22T03:33:46.773624+0000","last_undegraded":"2024-09-22T03:33:46.773624+0000","last_fullsized":"2024-09-22T03:33:46.773624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"84'769","reported_seq":794,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.761835+0000","last_change":"2024-09-22T03:32:12.899558+0000","last_active":"2024-09-22T03:33:46.761835+0000","last_peered":"2024-09-22T03:33:46.761835+0000","last_clean":"2024-09-22T03:33:46.761835+0000","last_became_active":"2024-09-22T03:32:12.893112+0000","last_became_peered":"2024-09-22T03:32:12.893112+0000","last_unstale":"2024-09-22T03:33:46.761835+0000","last_undegraded":"2024-09-22T03:33:46.761835+0000","last_fullsized":"2024-09-22T03:33:46.761835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":770,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":355,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.765612+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:32:45.765612+0000","last_peered":"2024-09-22T03:32:45.765612+0000","last_clean":"2024-09-22T03:32:45.765612+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:32:45.765612+0000","last_undegraded":"2024-09-22T03:32:45.765612+0000","last_fullsized":"2024-09-22T03:32:45.765612+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":221,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.779889+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:32:45.779889+0000","last_peered":"2024-09-22T03:32:45.779889+0000","last_clean":"2024-09-22T03:32:45.779889+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:32:45.779889+0000","last_undegraded":"2024-09-22T03:32:45.779889+0000","last_fullsized":"2024-09-22T03:32:45.779889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"84'855","reported_seq":880,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.876382+0000","last_change":"2024-09-22T03:32:12.901661+0000","last_active":"2024-09-22T03:33:46.876382+0000","last_peered":"2024-09-22T03:33:46.876382+0000","last_clean":"2024-09-22T03:33:46.876382+0000","last_became_active":"2024-09-22T03:32:12.893095+0000","last_became_peered":"2024-09-22T03:32:12.893095+0000","last_unstale":"2024-09-22T03:33:46.876382+0000","last_undegraded":"2024-09-22T03:33:46.876382+0000","last_fullsized":"2024-09-22T03:33:46.876382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.782955+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:32:45.782955+0000","last_peered":"2024-09-22T03:32:45.782955+0000","last_clean":"2024-09-22T03:32:45.782955+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:32:45.782955+0000","last_undegraded":"2024-09-22T03:32:45.782955+0000","last_fullsized":"2024-09-22T03:32:45.782955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"84'940","reported_seq":965,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.625759+0000","last_change":"2024-09-22T03:32:12.932433+0000","last_active":"2024-09-22T03:33:46.625759+0000","last_peered":"2024-09-22T03:33:46.625759+0000","last_clean":"2024-09-22T03:33:46.625759+0000","last_became_active":"2024-09-22T03:32:12.903794+0000","last_became_peered":"2024-09-22T03:32:12.903794+0000","last_unstale":"2024-09-22T03:33:46.625759+0000","last_undegraded":"2024-09-22T03:33:46.625759+0000","last_fullsized":"2024-09-22T03:33:46.625759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6086656,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":941,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.776397+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.776397+0000","last_peered":"2024-09-22T03:32:45.776397+0000","last_clean":"2024-09-22T03:32:45.776397+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:32:45.776397+0000","last_undegraded":"2024-09-22T03:32:45.776397+0000","last_fullsized":"2024-09-22T03:32:45.776397+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"84'699","reported_seq":724,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.750869+0000","last_change":"2024-09-22T03:32:12.949094+0000","last_active":"2024-09-22T03:33:46.750869+0000","last_peered":"2024-09-22T03:33:46.750869+0000","last_clean":"2024-09-22T03:33:46.750869+0000","last_became_active":"2024-09-22T03:32:12.943413+0000","last_became_peered":"2024-09-22T03:32:12.943413+0000","last_unstale":"2024-09-22T03:33:46.750869+0000","last_undegraded":"2024-09-22T03:33:46.750869+0000","last_fullsized":"2024-09-22T03:33:46.750869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":700,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":222,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.778581+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:32:45.778581+0000","last_peered":"2024-09-22T03:32:45.778581+0000","last_clean":"2024-09-22T03:32:45.778581+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:32:45.778581+0000","last_undegraded":"2024-09-22T03:32:45.778581+0000","last_fullsized":"2024-09-22T03:32:45.778581+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"84'738","reported_seq":763,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.937878+0000","last_change":"2024-09-22T03:32:12.967242+0000","last_active":"2024-09-22T03:33:46.937878+0000","last_peered":"2024-09-22T03:33:46.937878+0000","last_clean":"2024-09-22T03:33:46.937878+0000","last_became_active":"2024-09-22T03:32:12.954518+0000","last_became_peered":"2024-09-22T03:32:12.954518+0000","last_unstale":"2024-09-22T03:33:46.937878+0000","last_undegraded":"2024-09-22T03:33:46.937878+0000","last_fullsized":"2024-09-22T03:33:46.937878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":739,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.772627+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.772627+0000","last_peered":"2024-09-22T03:32:45.772627+0000","last_clean":"2024-09-22T03:32:45.772627+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:32:45.772627+0000","last_undegraded":"2024-09-22T03:32:45.772627+0000","last_fullsized":"2024-09-22T03:32:45.772627+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"84'891","reported_seq":916,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.681330+0000","last_change":"2024-09-22T03:32:12.904164+0000","last_active":"2024-09-22T03:33:46.681330+0000","last_peered":"2024-09-22T03:33:46.681330+0000","last_clean":"2024-09-22T03:33:46.681330+0000","last_became_active":"2024-09-22T03:32:12.893821+0000","last_became_peered":"2024-09-22T03:32:12.893821+0000","last_unstale":"2024-09-22T03:33:46.681330+0000","last_undegraded":"2024-09-22T03:33:46.681330+0000","last_fullsized":"2024-09-22T03:33:46.681330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":892,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.769530+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:32:45.769530+0000","last_peered":"2024-09-22T03:32:45.769530+0000","last_clean":"2024-09-22T03:32:45.769530+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:32:45.769530+0000","last_undegraded":"2024-09-22T03:32:45.769530+0000","last_fullsized":"2024-09-22T03:32:45.769530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"84'820","reported_seq":845,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.849740+0000","last_change":"2024-09-22T03:32:12.949059+0000","last_active":"2024-09-22T03:33:46.849740+0000","last_peered":"2024-09-22T03:33:46.849740+0000","last_clean":"2024-09-22T03:33:46.849740+0000","last_became_active":"2024-09-22T03:32:12.943449+0000","last_became_peered":"2024-09-22T03:32:12.943449+0000","last_unstale":"2024-09-22T03:33:46.849740+0000","last_undegraded":"2024-09-22T03:33:46.849740+0000","last_fullsized":"2024-09-22T03:33:46.849740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":821,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.771834+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:32:45.771834+0000","last_peered":"2024-09-22T03:32:45.771834+0000","last_clean":"2024-09-22T03:32:45.771834+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:32:45.771834+0000","last_undegraded":"2024-09-22T03:32:45.771834+0000","last_fullsized":"2024-09-22T03:32:45.771834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"84'931","reported_seq":957,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:33:46.882578+0000","last_change":"2024-09-22T03:32:12.938762+0000","last_active":"2024-09-22T03:33:46.882578+0000","last_peered":"2024-09-22T03:33:46.882578+0000","last_clean":"2024-09-22T03:33:46.882578+0000","last_became_active":"2024-09-22T03:32:12.931783+0000","last_became_peered":"2024-09-22T03:32:12.931783+0000","last_unstale":"2024-09-22T03:33:46.882578+0000","last_undegraded":"2024-09-22T03:33:46.882578+0000","last_fullsized":"2024-09-22T03:33:46.882578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:32:11.770328+0000","last_clean_scrub_stamp":"2024-09-22T03:32:11.770328+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225944,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":932,"num_write_kb":7297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-22T03:32:45.781290+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:32:45.781290+0000","last_peered":"2024-09-22T03:32:45.781290+0000","last_clean":"2024-09-22T03:32:45.781290+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:32:45.781290+0000","last_undegraded":"2024-09-22T03:32:45.781290+0000","last_fullsized":"2024-09-22T03:32:45.781290+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":82108440,"num_objects":1254,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13102,"num_write_kb":102073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213682294784,"internally_reserved":0,"allocated":529199104,"data_stored":529199104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13086,"ondisk_log_size":13086,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213682294784,"internally_reserved":0,"allocated":529199104,"data_stored":529199104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53423693824,"internally_reserved":0,"allocated":129179648,"data_stored":129179648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":139240,"kb_used_data":139240,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52158488,"statfs":{"total":53552873472,"available":53410291712,"internally_reserved":0,"allocated":142581760,"data_stored":142581760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":112476,"kb_used_data":112476,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52185252,"statfs":{"total":53552873472,"available":53437698048,"internally_reserved":0,"allocated":115175424,"data_stored":115175424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":165692,"kb_used_data":165692,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52132036,"statfs":{"total":53552873472,"available":53383204864,"internally_reserved":0,"allocated":169668608,"data_stored":169668608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":149912,"kb_used_data":149912,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52147816,"statfs":{"total":53552873472,"available":53399363584,"internally_reserved":0,"allocated":153509888,"data_stored":153509888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53423693824,"internally_reserved":0,"allocated":129179648,"data_stored":129179648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53411987456,"internally_reserved":0,"allocated":140886016,"data_stored":140886016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53399252992,"internally_reserved":0,"allocated":153620480,"data_stored":153620480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53447360512,"internally_reserved":0,"allocated":105512960,"data_stored":105512960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53423693824,"internally_reserved":0,"allocated":129179648,"data_stored":129179648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":53552873472,"available":53411987456,"internally_reserved":0,"allocated":140886016,"data_stored":140886016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":53552873472,"available":53399252992,"internally_reserved":0,"allocated":153620480,"data_stored":153620480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":53552873472,"available":53447360512,"internally_reserved":0,"allocated":105512960,"data_stored":105512960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":53552873472,"available":53423693824,"internally_reserved":0,"allocated":129179648,"data_stored":129179648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:33:49.847 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:49.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:49.993+0000 7f478d30e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:50.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:49.999+0000 7f478d30e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:52.181 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:52.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:52.337+0000 7f54b5e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:52.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:52.345+0000 7f54b5e4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:54.449 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:33:54.449 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:33:54.449 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:33:54.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:33:54.556 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:54.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:54.596+0000 7f976562c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:54.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:54.612+0000 7f976562c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:54.697 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:54.697 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:54.698 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:54.702 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:54.703 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:54.703 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:33:54.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:54.711+0000 7fce79cd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:54.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:54.719+0000 7fce79cd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:54.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:54.872+0000 7ff81eeba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:54.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:54.886+0000 7ff81eeba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:54.983 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:54.984 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:54.988 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:54.988 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:54.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:33:55.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:55.167+0000 7fbcd82ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:55.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:55.182+0000 7fbcd82ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:55.207 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:55.211 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:55.212 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:55.212 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:33:55.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:55.333+0000 7fd8d410e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:55.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:55.342+0000 7fd8d410e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:55.358 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:33:55.358 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:33:55.359 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:33:55.362 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:33:55.363 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:33:55.591 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 1595 objects 2024-09-22T03:33:55.591 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :10.6887 2024-09-22T03:33:57.073 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:33:57.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:57.175+0000 7f9a1822c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:57.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:57.180+0000 7f9a1822c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:57.525 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_5 2024-09-22T03:33:57.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_5 unique_pool_5 --yes-i-really-really-mean-it 2024-09-22T03:33:57.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:57.626+0000 7f0f22357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:57.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:57.637+0000 7f0f22357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:58.113 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_5' removed 2024-09-22T03:33:58.130 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:33:58.130 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:33:58.130 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_6 2024-09-22T03:33:58.130 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_6 16 2024-09-22T03:33:58.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:58.267+0000 7fc826317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:58.280+0000 7fc826317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:59.114 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_6' created 2024-09-22T03:33:59.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_6 rados --yes-i-really-mean-it || true 2024-09-22T03:33:59.256 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:33:59.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:59.252+0000 7f2de3757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:59.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:59.264+0000 7f2de3757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:59.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:59.357+0000 7fb04e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:33:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:33:59.361+0000 7fb04e197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.121 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_6' 2024-09-22T03:34:00.363 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:00.363 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:00.364 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:34:00.364 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:34:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.465+0000 7f0186023640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.473+0000 7f0186023640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:00.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:00.490 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:00.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:34:00.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.589+0000 7f96ae797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.597+0000 7f96ae797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.611 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:00.611 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:00.611 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:00.611 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:00.612 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:00.614 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:00.615 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:00.615 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:34:00.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.713+0000 7fc461d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.720+0000 7fc461d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.734 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:00.734 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:00.734 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:00.735 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:00.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:00.738 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:00.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:34:00.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.839+0000 7f2110b4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:00.847+0000 7f2110b4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:00.862 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:00.863 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:00.863 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:00.865 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:00.865 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:01.141 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_6 bench 90 write' 2024-09-22T03:34:01.143 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:34:01.143 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:34:01.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:34:01.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:34:01.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_58401 2024-09-22T03:34:01.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:34:01.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:34:01.433 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:01.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:01.549+0000 7f9dc4cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:01.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:01.554+0000 7f9dc4cb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:02.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 148 132 1.03116 1.03125 0.0199652 0.108889 2024-09-22T03:34:03.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 295 279 1.08975 1.14844 0.237015 0.108304 2024-09-22T03:34:03.654 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:03.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:03.795+0000 7f4a2526c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:03.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:03.801+0000 7f4a2526c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:04.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 456 440 1.14575 1.25781 0.0715289 0.106953 2024-09-22T03:34:05.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 608 592 1.15617 1.1875 0.0371575 0.105436 2024-09-22T03:34:05.866 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:05.867 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:05.867 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:34:05.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:06.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:06.011+0000 7f16ed04f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:06.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:06.023+0000 7f16ed04f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:06.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:06.084+0000 7f6efb26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:06.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:06.090+0000 7f6efb26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:06.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 756 740 1.15616 1.15625 0.0557467 0.106341 2024-09-22T03:34:06.315 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:34:06.315 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:34:06.339 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-22T03:34:06.058608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1524275,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":253,"num_write_kb":2808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":11336,"kb_used_data":11336,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209179576,"statfs":{"total":214211493888,"available":214199885824,"internally_reserved":0,"allocated":11608064,"data_stored":11608064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":933888,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.944067"},"pg_stats":[{"pgid":"9.7","version":"88'23","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.747867+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:34:01.747867+0000","last_peered":"2024-09-22T03:34:01.747867+0000","last_clean":"2024-09-22T03:34:01.747867+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:34:01.747867+0000","last_undegraded":"2024-09-22T03:34:01.747867+0000","last_fullsized":"2024-09-22T03:34:01.747867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'15","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.480555+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:34:01.480555+0000","last_peered":"2024-09-22T03:34:01.480555+0000","last_clean":"2024-09-22T03:34:01.480555+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:34:01.480555+0000","last_undegraded":"2024-09-22T03:34:01.480555+0000","last_fullsized":"2024-09-22T03:34:01.480555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147926+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:34:01.147926+0000","last_peered":"2024-09-22T03:34:01.147926+0000","last_clean":"2024-09-22T03:34:01.147926+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:34:01.147926+0000","last_undegraded":"2024-09-22T03:34:01.147926+0000","last_fullsized":"2024-09-22T03:34:01.147926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'7","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.954582+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:34:01.954582+0000","last_peered":"2024-09-22T03:34:01.954582+0000","last_clean":"2024-09-22T03:34:01.954582+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:34:01.954582+0000","last_undegraded":"2024-09-22T03:34:01.954582+0000","last_fullsized":"2024-09-22T03:34:01.954582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'17","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.970513+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:34:01.970513+0000","last_peered":"2024-09-22T03:34:01.970513+0000","last_clean":"2024-09-22T03:34:01.970513+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:34:01.970513+0000","last_undegraded":"2024-09-22T03:34:01.970513+0000","last_fullsized":"2024-09-22T03:34:01.970513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141126+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:34:01.141126+0000","last_peered":"2024-09-22T03:34:01.141126+0000","last_clean":"2024-09-22T03:34:01.141126+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:34:01.141126+0000","last_undegraded":"2024-09-22T03:34:01.141126+0000","last_fullsized":"2024-09-22T03:34:01.141126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'7","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.300829+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:34:01.300829+0000","last_peered":"2024-09-22T03:34:01.300829+0000","last_clean":"2024-09-22T03:34:01.300829+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:34:01.300829+0000","last_undegraded":"2024-09-22T03:34:01.300829+0000","last_fullsized":"2024-09-22T03:34:01.300829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'15","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.814218+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:34:01.814218+0000","last_peered":"2024-09-22T03:34:01.814218+0000","last_clean":"2024-09-22T03:34:01.814218+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:34:01.814218+0000","last_undegraded":"2024-09-22T03:34:01.814218+0000","last_fullsized":"2024-09-22T03:34:01.814218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141799+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:34:01.141799+0000","last_peered":"2024-09-22T03:34:01.141799+0000","last_clean":"2024-09-22T03:34:01.141799+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:34:01.141799+0000","last_undegraded":"2024-09-22T03:34:01.141799+0000","last_fullsized":"2024-09-22T03:34:01.141799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143167+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:34:01.143167+0000","last_peered":"2024-09-22T03:34:01.143167+0000","last_clean":"2024-09-22T03:34:01.143167+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:34:01.143167+0000","last_undegraded":"2024-09-22T03:34:01.143167+0000","last_fullsized":"2024-09-22T03:34:01.143167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'7","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.530714+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:34:01.530714+0000","last_peered":"2024-09-22T03:34:01.530714+0000","last_clean":"2024-09-22T03:34:01.530714+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:34:01.530714+0000","last_undegraded":"2024-09-22T03:34:01.530714+0000","last_fullsized":"2024-09-22T03:34:01.530714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145729+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:34:01.145729+0000","last_peered":"2024-09-22T03:34:01.145729+0000","last_clean":"2024-09-22T03:34:01.145729+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:34:01.145729+0000","last_undegraded":"2024-09-22T03:34:01.145729+0000","last_fullsized":"2024-09-22T03:34:01.145729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143767+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:34:01.143767+0000","last_peered":"2024-09-22T03:34:01.143767+0000","last_clean":"2024-09-22T03:34:01.143767+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:34:01.143767+0000","last_undegraded":"2024-09-22T03:34:01.143767+0000","last_fullsized":"2024-09-22T03:34:01.143767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'15","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.902276+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:34:01.902276+0000","last_peered":"2024-09-22T03:34:01.902276+0000","last_clean":"2024-09-22T03:34:01.902276+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:34:01.902276+0000","last_undegraded":"2024-09-22T03:34:01.902276+0000","last_fullsized":"2024-09-22T03:34:01.902276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.144366+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:34:01.144366+0000","last_peered":"2024-09-22T03:34:01.144366+0000","last_clean":"2024-09-22T03:34:01.144366+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:34:01.144366+0000","last_undegraded":"2024-09-22T03:34:01.144366+0000","last_fullsized":"2024-09-22T03:34:01.144366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.142380+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:34:01.142380+0000","last_peered":"2024-09-22T03:34:01.142380+0000","last_clean":"2024-09-22T03:34:01.142380+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:34:01.142380+0000","last_undegraded":"2024-09-22T03:34:01.142380+0000","last_fullsized":"2024-09-22T03:34:01.142380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":933888,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194442240,"internally_reserved":0,"allocated":17051648,"data_stored":17051648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194515968,"internally_reserved":0,"allocated":16977920,"data_stored":16977920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53548466176,"internally_reserved":0,"allocated":4407296,"data_stored":4407296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706323,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2952,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294776,"statfs":{"total":53552873472,"available":53549850624,"internally_reserved":0,"allocated":3022848,"data_stored":3022848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706323,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2936,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294792,"statfs":{"total":53552873472,"available":53549867008,"internally_reserved":0,"allocated":3006464,"data_stored":3006464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706323,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2020,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295708,"statfs":{"total":53552873472,"available":53550804992,"internally_reserved":0,"allocated":2068480,"data_stored":2068480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706323,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3428,"kb_used_data":3428,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294300,"statfs":{"total":53552873472,"available":53549363200,"internally_reserved":0,"allocated":3510272,"data_stored":3510272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53548466176,"internally_reserved":0,"allocated":4407296,"data_stored":4407296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53549490176,"internally_reserved":0,"allocated":3383296,"data_stored":3383296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53548654592,"internally_reserved":0,"allocated":4218880,"data_stored":4218880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53548466176,"internally_reserved":0,"allocated":4407296,"data_stored":4407296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53547831296,"internally_reserved":0,"allocated":5042176,"data_stored":5042176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53549490176,"internally_reserved":0,"allocated":3383296,"data_stored":3383296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53548654592,"internally_reserved":0,"allocated":4218880,"data_stored":4218880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53548466176,"internally_reserved":0,"allocated":4407296,"data_stored":4407296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:34:07.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 910 894 1.16398 1.20312 0.0995115 0.106157 2024-09-22T03:34:08.199 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 1027 1011 1.12826 0.914062 0.174241 0.110195 2024-09-22T03:34:08.316 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:08.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:08.443+0000 7f3aab117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:08.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:08.448+0000 7f3aab117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:09.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 16 1184 1168 1.14053 1.22656 0.0434356 0.108521 2024-09-22T03:34:10.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1338 1322 1.14747 1.20312 0.0525103 0.108004 2024-09-22T03:34:10.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:10.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:10.677+0000 7fb923f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:10.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:10.684+0000 7fb923f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:11.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 1462 1446 1.12959 0.96875 0.148896 0.109966 2024-09-22T03:34:11.341 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:11.341 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:11.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:34:11.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:11.484+0000 7f540142c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:11.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:11.493+0000 7f540142c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:11.835 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:34:11.835 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:34:11.848 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-22T03:34:10.059375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7717427,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1009,"num_write_kb":8856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":954,"ondisk_log_size":954,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":47388,"kb_used_data":47388,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209143524,"statfs":{"total":214211493888,"available":214162968576,"internally_reserved":0,"allocated":48525312,"data_stored":48525312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7127040,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":870,"num_write_kb":6960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.938356"},"pg_stats":[{"pgid":"9.7","version":"88'63","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:05.962607+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:34:05.962607+0000","last_peered":"2024-09-22T03:34:05.962607+0000","last_clean":"2024-09-22T03:34:05.962607+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:34:05.962607+0000","last_undegraded":"2024-09-22T03:34:05.962607+0000","last_fullsized":"2024-09-22T03:34:05.962607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'43","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.975107+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:34:06.975107+0000","last_peered":"2024-09-22T03:34:06.975107+0000","last_clean":"2024-09-22T03:34:06.975107+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:34:06.975107+0000","last_undegraded":"2024-09-22T03:34:06.975107+0000","last_fullsized":"2024-09-22T03:34:06.975107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'23","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.171494+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:34:06.171494+0000","last_peered":"2024-09-22T03:34:06.171494+0000","last_clean":"2024-09-22T03:34:06.171494+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:34:06.171494+0000","last_undegraded":"2024-09-22T03:34:06.171494+0000","last_fullsized":"2024-09-22T03:34:06.171494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'63","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.526455+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:34:06.526455+0000","last_peered":"2024-09-22T03:34:06.526455+0000","last_clean":"2024-09-22T03:34:06.526455+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:34:06.526455+0000","last_undegraded":"2024-09-22T03:34:06.526455+0000","last_fullsized":"2024-09-22T03:34:06.526455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'63","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:04.731762+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:34:04.731762+0000","last_peered":"2024-09-22T03:34:04.731762+0000","last_clean":"2024-09-22T03:34:04.731762+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:34:04.731762+0000","last_undegraded":"2024-09-22T03:34:04.731762+0000","last_fullsized":"2024-09-22T03:34:04.731762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'79","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.776487+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:34:06.776487+0000","last_peered":"2024-09-22T03:34:06.776487+0000","last_clean":"2024-09-22T03:34:06.776487+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:34:06.776487+0000","last_undegraded":"2024-09-22T03:34:06.776487+0000","last_fullsized":"2024-09-22T03:34:06.776487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'47","reported_seq":60,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.579646+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:34:06.579646+0000","last_peered":"2024-09-22T03:34:06.579646+0000","last_clean":"2024-09-22T03:34:06.579646+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:34:06.579646+0000","last_undegraded":"2024-09-22T03:34:06.579646+0000","last_fullsized":"2024-09-22T03:34:06.579646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'73","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.992520+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:34:06.992520+0000","last_peered":"2024-09-22T03:34:06.992520+0000","last_clean":"2024-09-22T03:34:06.992520+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:34:06.992520+0000","last_undegraded":"2024-09-22T03:34:06.992520+0000","last_fullsized":"2024-09-22T03:34:06.992520+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":606208,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'23","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.441049+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:34:06.441049+0000","last_peered":"2024-09-22T03:34:06.441049+0000","last_clean":"2024-09-22T03:34:06.441049+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:34:06.441049+0000","last_undegraded":"2024-09-22T03:34:06.441049+0000","last_fullsized":"2024-09-22T03:34:06.441049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'71","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.057641+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:34:06.057641+0000","last_peered":"2024-09-22T03:34:06.057641+0000","last_clean":"2024-09-22T03:34:06.057641+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:34:06.057641+0000","last_undegraded":"2024-09-22T03:34:06.057641+0000","last_fullsized":"2024-09-22T03:34:06.057641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'87","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.956388+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:34:06.956388+0000","last_peered":"2024-09-22T03:34:06.956388+0000","last_clean":"2024-09-22T03:34:06.956388+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:34:06.956388+0000","last_undegraded":"2024-09-22T03:34:06.956388+0000","last_fullsized":"2024-09-22T03:34:06.956388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"88'15","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.309635+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:34:06.309635+0000","last_peered":"2024-09-22T03:34:06.309635+0000","last_clean":"2024-09-22T03:34:06.309635+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:34:06.309635+0000","last_undegraded":"2024-09-22T03:34:06.309635+0000","last_fullsized":"2024-09-22T03:34:06.309635+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'55","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.846258+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:34:06.846258+0000","last_peered":"2024-09-22T03:34:06.846258+0000","last_clean":"2024-09-22T03:34:06.846258+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:34:06.846258+0000","last_undegraded":"2024-09-22T03:34:06.846258+0000","last_fullsized":"2024-09-22T03:34:06.846258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'71","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.918392+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:34:06.918392+0000","last_peered":"2024-09-22T03:34:06.918392+0000","last_clean":"2024-09-22T03:34:06.918392+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:34:06.918392+0000","last_undegraded":"2024-09-22T03:34:06.918392+0000","last_fullsized":"2024-09-22T03:34:06.918392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'39","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:05.829319+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:34:05.829319+0000","last_peered":"2024-09-22T03:34:05.829319+0000","last_clean":"2024-09-22T03:34:05.829319+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:34:05.829319+0000","last_undegraded":"2024-09-22T03:34:05.829319+0000","last_fullsized":"2024-09-22T03:34:05.829319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"88'39","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:06.454320+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:34:06.454320+0000","last_peered":"2024-09-22T03:34:06.454320+0000","last_clean":"2024-09-22T03:34:06.454320+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:34:06.454320+0000","last_undegraded":"2024-09-22T03:34:06.454320+0000","last_fullsized":"2024-09-22T03:34:06.454320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":7127040,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":870,"num_write_kb":6960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214157328384,"internally_reserved":0,"allocated":54165504,"data_stored":54165504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214157344768,"internally_reserved":0,"allocated":54149120,"data_stored":54149120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53539319808,"internally_reserved":0,"allocated":13553664,"data_stored":13553664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":12140,"kb_used_data":12140,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52285588,"statfs":{"total":53552873472,"available":53540442112,"internally_reserved":0,"allocated":12431360,"data_stored":12431360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":10688,"kb_used_data":10688,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52287040,"statfs":{"total":53552873472,"available":53541928960,"internally_reserved":0,"allocated":10944512,"data_stored":10944512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":9612,"kb_used_data":9612,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52288116,"statfs":{"total":53552873472,"available":53543030784,"internally_reserved":0,"allocated":9842688,"data_stored":9842688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706328,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":14948,"kb_used_data":14948,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52282780,"statfs":{"total":53552873472,"available":53537566720,"internally_reserved":0,"allocated":15306752,"data_stored":15306752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53539319808,"internally_reserved":0,"allocated":13553664,"data_stored":13553664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53535834112,"internally_reserved":0,"allocated":17039360,"data_stored":17039360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53541474304,"internally_reserved":0,"allocated":11399168,"data_stored":11399168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53540716544,"internally_reserved":0,"allocated":12156928,"data_stored":12156928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53539319808,"internally_reserved":0,"allocated":13553664,"data_stored":13553664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53535834112,"internally_reserved":0,"allocated":17039360,"data_stored":17039360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53541466112,"internally_reserved":0,"allocated":11407360,"data_stored":11407360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53540708352,"internally_reserved":0,"allocated":12165120,"data_stored":12165120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53539319808,"internally_reserved":0,"allocated":13553664,"data_stored":13553664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:34:12.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 1618 1602 1.13769 1.21875 0.0931209 0.109326 2024-09-22T03:34:12.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:12.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:12.887+0000 7f0789c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:12.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:12.892+0000 7f0789c58640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:13.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 1772 1756 1.14313 1.20312 0.129463 0.108753 2024-09-22T03:34:14.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 1941 1925 1.15675 1.32031 0.0816588 0.107599 2024-09-22T03:34:15.048 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:15.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:15.175+0000 7f5026d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:15.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:15.182+0000 7f5026d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:15.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 2097 2081 1.16118 1.21875 0.0457725 0.106896 2024-09-22T03:34:16.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 2240 2224 1.15824 1.11719 0.0902796 0.107547 2024-09-22T03:34:16.850 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:16.850 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:16.851 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:34:16.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:16.987+0000 7fa321ab2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:16.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:16.997+0000 7fa321ab2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:17.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 2398 2382 1.16299 1.23438 0.134731 0.106999 2024-09-22T03:34:17.300 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:34:17.301 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:34:17.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:17.325 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-22T03:34:16.060454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13525555,"num_objects":202,"num_object_clones":0,"num_object_copies":606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1718,"num_write_kb":14528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1663,"ondisk_log_size":1663,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":81544,"kb_used_data":81544,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209109368,"statfs":{"total":214211493888,"available":214127992832,"internally_reserved":0,"allocated":83501056,"data_stored":83501056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12001280,"num_objects":183,"num_object_clones":0,"num_object_copies":549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":11720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002132"},"pg_stats":[{"pgid":"9.7","version":"88'111","reported_seq":124,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:10.172782+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:34:10.172782+0000","last_peered":"2024-09-22T03:34:10.172782+0000","last_clean":"2024-09-22T03:34:10.172782+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:34:10.172782+0000","last_undegraded":"2024-09-22T03:34:10.172782+0000","last_fullsized":"2024-09-22T03:34:10.172782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'95","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.919566+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:34:11.919566+0000","last_peered":"2024-09-22T03:34:11.919566+0000","last_clean":"2024-09-22T03:34:11.919566+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:34:11.919566+0000","last_undegraded":"2024-09-22T03:34:11.919566+0000","last_fullsized":"2024-09-22T03:34:11.919566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'34","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.999265+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:34:11.999265+0000","last_peered":"2024-09-22T03:34:11.999265+0000","last_clean":"2024-09-22T03:34:11.999265+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:34:11.999265+0000","last_undegraded":"2024-09-22T03:34:11.999265+0000","last_fullsized":"2024-09-22T03:34:11.999265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":286720,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'79","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:10.135666+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:34:10.135666+0000","last_peered":"2024-09-22T03:34:10.135666+0000","last_clean":"2024-09-22T03:34:10.135666+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:34:10.135666+0000","last_undegraded":"2024-09-22T03:34:10.135666+0000","last_fullsized":"2024-09-22T03:34:10.135666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'119","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.814054+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:34:11.814054+0000","last_peered":"2024-09-22T03:34:11.814054+0000","last_clean":"2024-09-22T03:34:11.814054+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:34:11.814054+0000","last_undegraded":"2024-09-22T03:34:11.814054+0000","last_fullsized":"2024-09-22T03:34:11.814054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'103","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.287989+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:34:11.287989+0000","last_peered":"2024-09-22T03:34:11.287989+0000","last_clean":"2024-09-22T03:34:11.287989+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:34:11.287989+0000","last_undegraded":"2024-09-22T03:34:11.287989+0000","last_fullsized":"2024-09-22T03:34:11.287989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'71","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.890588+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:34:11.890588+0000","last_peered":"2024-09-22T03:34:11.890588+0000","last_clean":"2024-09-22T03:34:11.890588+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:34:11.890588+0000","last_undegraded":"2024-09-22T03:34:11.890588+0000","last_fullsized":"2024-09-22T03:34:11.890588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'111","reported_seq":124,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.109400+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:34:11.109400+0000","last_peered":"2024-09-22T03:34:11.109400+0000","last_clean":"2024-09-22T03:34:11.109400+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:34:11.109400+0000","last_undegraded":"2024-09-22T03:34:11.109400+0000","last_fullsized":"2024-09-22T03:34:11.109400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'47","reported_seq":60,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.878124+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:34:11.878124+0000","last_peered":"2024-09-22T03:34:11.878124+0000","last_clean":"2024-09-22T03:34:11.878124+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:34:11.878124+0000","last_undegraded":"2024-09-22T03:34:11.878124+0000","last_fullsized":"2024-09-22T03:34:11.878124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'143","reported_seq":156,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.557342+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:34:11.557342+0000","last_peered":"2024-09-22T03:34:11.557342+0000","last_clean":"2024-09-22T03:34:11.557342+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:34:11.557342+0000","last_undegraded":"2024-09-22T03:34:11.557342+0000","last_fullsized":"2024-09-22T03:34:11.557342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'143","reported_seq":156,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.343717+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:34:11.343717+0000","last_peered":"2024-09-22T03:34:11.343717+0000","last_clean":"2024-09-22T03:34:11.343717+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:34:11.343717+0000","last_undegraded":"2024-09-22T03:34:11.343717+0000","last_fullsized":"2024-09-22T03:34:11.343717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"88'63","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.471853+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:34:11.471853+0000","last_peered":"2024-09-22T03:34:11.471853+0000","last_clean":"2024-09-22T03:34:11.471853+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:34:11.471853+0000","last_undegraded":"2024-09-22T03:34:11.471853+0000","last_fullsized":"2024-09-22T03:34:11.471853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'175","reported_seq":188,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.971021+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:34:11.971021+0000","last_peered":"2024-09-22T03:34:11.971021+0000","last_clean":"2024-09-22T03:34:11.971021+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:34:11.971021+0000","last_undegraded":"2024-09-22T03:34:11.971021+0000","last_fullsized":"2024-09-22T03:34:11.971021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'87","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:08.977253+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:34:08.977253+0000","last_peered":"2024-09-22T03:34:08.977253+0000","last_clean":"2024-09-22T03:34:08.977253+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:34:08.977253+0000","last_undegraded":"2024-09-22T03:34:08.977253+0000","last_fullsized":"2024-09-22T03:34:08.977253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'103","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.223038+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:34:11.223038+0000","last_peered":"2024-09-22T03:34:11.223038+0000","last_clean":"2024-09-22T03:34:11.223038+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:34:11.223038+0000","last_undegraded":"2024-09-22T03:34:11.223038+0000","last_fullsized":"2024-09-22T03:34:11.223038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"88'79","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:11.582998+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:34:11.582998+0000","last_peered":"2024-09-22T03:34:11.582998+0000","last_clean":"2024-09-22T03:34:11.582998+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:34:11.582998+0000","last_undegraded":"2024-09-22T03:34:11.582998+0000","last_fullsized":"2024-09-22T03:34:11.582998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":12935168,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1579,"num_write_kb":12632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214122008576,"internally_reserved":0,"allocated":89485312,"data_stored":89485312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1563,"ondisk_log_size":1563,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214122008576,"internally_reserved":0,"allocated":89485312,"data_stored":89485312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53528907776,"internally_reserved":0,"allocated":23965696,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":21516,"kb_used_data":21516,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52276212,"statfs":{"total":53552873472,"available":53530841088,"internally_reserved":0,"allocated":22032384,"data_stored":22032384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":16964,"kb_used_data":16964,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52280764,"statfs":{"total":53552873472,"available":53535502336,"internally_reserved":0,"allocated":17371136,"data_stored":17371136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":17396,"kb_used_data":17396,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52280332,"statfs":{"total":53552873472,"available":53535059968,"internally_reserved":0,"allocated":17813504,"data_stored":17813504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":25668,"kb_used_data":25668,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52272060,"statfs":{"total":53552873472,"available":53526589440,"internally_reserved":0,"allocated":26284032,"data_stored":26284032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53528907776,"internally_reserved":0,"allocated":23965696,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53524918272,"internally_reserved":0,"allocated":27955200,"data_stored":27955200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53533790208,"internally_reserved":0,"allocated":19083264,"data_stored":19083264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53534392320,"internally_reserved":0,"allocated":18481152,"data_stored":18481152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53528907776,"internally_reserved":0,"allocated":23965696,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53524918272,"internally_reserved":0,"allocated":27955200,"data_stored":27955200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53533790208,"internally_reserved":0,"allocated":19083264,"data_stored":19083264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53534392320,"internally_reserved":0,"allocated":18481152,"data_stored":18481152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53528907776,"internally_reserved":0,"allocated":23965696,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:34:17.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:17.446+0000 7f6b334a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:17.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:17.452+0000 7f6b334a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:18.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 2536 2520 1.15799 1.07812 0.398535 0.107183 2024-09-22T03:34:19.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 2680 2664 1.15615 1.125 0.0657965 0.10755 2024-09-22T03:34:19.625 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:19.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:19.773+0000 7f60c1b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:19.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:19.780+0000 7f60c1b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:20.200 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 2832 2816 1.1578 1.1875 0.108867 0.107616 2024-09-22T03:34:21.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:34:21.200560+0000 min lat: 0.0135566 max lat: 0.501871 avg lat: 0.107088 2024-09-22T03:34:21.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:34:21.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 2993 2977 1.16279 1.25781 0.118899 0.107088 2024-09-22T03:34:21.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:22.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:22.067+0000 7f6f68957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:22.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:22.073+0000 7f6f68957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:22.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 3136 3120 1.16062 1.11719 0.139674 0.107366 2024-09-22T03:34:22.328 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:22.328 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:22.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:34:22.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:22.474+0000 7f72dded7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:22.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:22.485+0000 7f72dded7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:22.679 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:34:22.679 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:34:22.693 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-22T03:34:22.061581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25985587,"num_objects":392,"num_object_clones":0,"num_object_copies":1176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":3239,"num_write_kb":26696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3184,"ondisk_log_size":3184,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":155852,"kb_used_data":155852,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209035060,"statfs":{"total":214211493888,"available":214051901440,"internally_reserved":0,"allocated":159592448,"data_stored":159592448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18268160,"num_objects":278,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2230,"num_write_kb":17840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002205"},"pg_stats":[{"pgid":"9.7","version":"88'167","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.550237+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:34:21.550237+0000","last_peered":"2024-09-22T03:34:21.550237+0000","last_clean":"2024-09-22T03:34:21.550237+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:34:21.550237+0000","last_undegraded":"2024-09-22T03:34:21.550237+0000","last_fullsized":"2024-09-22T03:34:21.550237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'199","reported_seq":212,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:20.534868+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:34:20.534868+0000","last_peered":"2024-09-22T03:34:20.534868+0000","last_clean":"2024-09-22T03:34:20.534868+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:34:20.534868+0000","last_undegraded":"2024-09-22T03:34:20.534868+0000","last_fullsized":"2024-09-22T03:34:20.534868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'167","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:20.943881+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:34:20.943881+0000","last_peered":"2024-09-22T03:34:20.943881+0000","last_clean":"2024-09-22T03:34:20.943881+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:34:20.943881+0000","last_undegraded":"2024-09-22T03:34:20.943881+0000","last_fullsized":"2024-09-22T03:34:20.943881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'183","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.785675+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:34:21.785675+0000","last_peered":"2024-09-22T03:34:21.785675+0000","last_clean":"2024-09-22T03:34:21.785675+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:34:21.785675+0000","last_undegraded":"2024-09-22T03:34:21.785675+0000","last_fullsized":"2024-09-22T03:34:21.785675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'183","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:19.057028+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:34:19.057028+0000","last_peered":"2024-09-22T03:34:19.057028+0000","last_clean":"2024-09-22T03:34:19.057028+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:34:19.057028+0000","last_undegraded":"2024-09-22T03:34:19.057028+0000","last_fullsized":"2024-09-22T03:34:19.057028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'167","reported_seq":180,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.653368+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:34:21.653368+0000","last_peered":"2024-09-22T03:34:21.653368+0000","last_clean":"2024-09-22T03:34:21.653368+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:34:21.653368+0000","last_undegraded":"2024-09-22T03:34:21.653368+0000","last_fullsized":"2024-09-22T03:34:21.653368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'171","reported_seq":184,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.985909+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:34:21.985909+0000","last_peered":"2024-09-22T03:34:21.985909+0000","last_clean":"2024-09-22T03:34:21.985909+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:34:21.985909+0000","last_undegraded":"2024-09-22T03:34:21.985909+0000","last_fullsized":"2024-09-22T03:34:21.985909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":1376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'191","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.979656+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:34:21.979656+0000","last_peered":"2024-09-22T03:34:21.979656+0000","last_clean":"2024-09-22T03:34:21.979656+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:34:21.979656+0000","last_undegraded":"2024-09-22T03:34:21.979656+0000","last_fullsized":"2024-09-22T03:34:21.979656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'119","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.571945+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:34:21.571945+0000","last_peered":"2024-09-22T03:34:21.571945+0000","last_clean":"2024-09-22T03:34:21.571945+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:34:21.571945+0000","last_undegraded":"2024-09-22T03:34:21.571945+0000","last_fullsized":"2024-09-22T03:34:21.571945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'255","reported_seq":268,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.467479+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:34:21.467479+0000","last_peered":"2024-09-22T03:34:21.467479+0000","last_clean":"2024-09-22T03:34:21.467479+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:34:21.467479+0000","last_undegraded":"2024-09-22T03:34:21.467479+0000","last_fullsized":"2024-09-22T03:34:21.467479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'247","reported_seq":260,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.861429+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:34:21.861429+0000","last_peered":"2024-09-22T03:34:21.861429+0000","last_clean":"2024-09-22T03:34:21.861429+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:34:21.861429+0000","last_undegraded":"2024-09-22T03:34:21.861429+0000","last_fullsized":"2024-09-22T03:34:21.861429+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"88'183","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.826058+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:34:21.826058+0000","last_peered":"2024-09-22T03:34:21.826058+0000","last_clean":"2024-09-22T03:34:21.826058+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:34:21.826058+0000","last_undegraded":"2024-09-22T03:34:21.826058+0000","last_fullsized":"2024-09-22T03:34:21.826058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'295","reported_seq":308,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.423472+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:34:21.423472+0000","last_peered":"2024-09-22T03:34:21.423472+0000","last_clean":"2024-09-22T03:34:21.423472+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:34:21.423472+0000","last_undegraded":"2024-09-22T03:34:21.423472+0000","last_fullsized":"2024-09-22T03:34:21.423472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'183","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:18.748721+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:34:18.748721+0000","last_peered":"2024-09-22T03:34:18.748721+0000","last_clean":"2024-09-22T03:34:18.748721+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:34:18.748721+0000","last_undegraded":"2024-09-22T03:34:18.748721+0000","last_fullsized":"2024-09-22T03:34:18.748721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'255","reported_seq":268,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:20.689124+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:34:20.689124+0000","last_peered":"2024-09-22T03:34:20.689124+0000","last_clean":"2024-09-22T03:34:20.689124+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:34:20.689124+0000","last_undegraded":"2024-09-22T03:34:20.689124+0000","last_fullsized":"2024-09-22T03:34:20.689124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":256,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"88'119","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:21.923772+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:34:21.923772+0000","last_peered":"2024-09-22T03:34:21.923772+0000","last_clean":"2024-09-22T03:34:21.923772+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:34:21.923772+0000","last_undegraded":"2024-09-22T03:34:21.923772+0000","last_fullsized":"2024-09-22T03:34:21.923772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":25395200,"num_objects":388,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":24800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214046314496,"internally_reserved":0,"allocated":165179392,"data_stored":165179392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3084,"ondisk_log_size":3084,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214046314496,"internally_reserved":0,"allocated":165179392,"data_stored":165179392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53507731456,"internally_reserved":0,"allocated":45142016,"data_stored":45142016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":42448,"kb_used_data":42448,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52255280,"statfs":{"total":53552873472,"available":53509406720,"internally_reserved":0,"allocated":43466752,"data_stored":43466752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":33656,"kb_used_data":33656,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52264072,"statfs":{"total":53552873472,"available":53518409728,"internally_reserved":0,"allocated":34463744,"data_stored":34463744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":31316,"kb_used_data":31316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52266412,"statfs":{"total":53552873472,"available":53520805888,"internally_reserved":0,"allocated":32067584,"data_stored":32067584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":48432,"kb_used_data":48432,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52249296,"statfs":{"total":53552873472,"available":53503279104,"internally_reserved":0,"allocated":49594368,"data_stored":49594368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53507731456,"internally_reserved":0,"allocated":45142016,"data_stored":45142016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53501493248,"internally_reserved":0,"allocated":51380224,"data_stored":51380224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53519708160,"internally_reserved":0,"allocated":33165312,"data_stored":33165312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53517389824,"internally_reserved":0,"allocated":35483648,"data_stored":35483648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53507723264,"internally_reserved":0,"allocated":45150208,"data_stored":45150208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53501493248,"internally_reserved":0,"allocated":51380224,"data_stored":51380224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53519708160,"internally_reserved":0,"allocated":33165312,"data_stored":33165312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53517389824,"internally_reserved":0,"allocated":35483648,"data_stored":35483648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53507723264,"internally_reserved":0,"allocated":45150208,"data_stored":45150208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:34:23.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 3292 3276 1.16325 1.21875 0.101833 0.107174 2024-09-22T03:34:24.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:24.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 3445 3429 1.16464 1.19531 0.0912997 0.107001 2024-09-22T03:34:24.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:24.317+0000 7f91d66b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:24.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:24.324+0000 7f91d66b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:25.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 3584 3568 1.16136 1.08594 0.122624 0.107265 2024-09-22T03:34:26.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 3736 3720 1.1624 1.1875 0.0941065 0.10725 2024-09-22T03:34:26.548 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:26.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:26.668+0000 7f01dacd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:26.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:26.674+0000 7f01dacd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:27.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 3901 3885 1.16727 1.28906 0.0845466 0.106794 2024-09-22T03:34:27.695 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:27.695 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:27.696 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:34:27.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:27.811+0000 7f454f597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:27.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:27.820+0000 7f454f597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:28.107 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:34:28.107 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:34:28.125 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-22T03:34:28.062788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":32277043,"num_objects":488,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":4007,"num_write_kb":32840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3952,"ondisk_log_size":3952,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":192132,"kb_used_data":192132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208998780,"statfs":{"total":214211493888,"available":214014750720,"internally_reserved":0,"allocated":196743168,"data_stored":196743168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18751488,"num_objects":286,"num_object_clones":0,"num_object_copies":858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2289,"num_write_kb":18312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002333"},"pg_stats":[{"pgid":"9.7","version":"88'223","reported_seq":236,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.586991+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:34:26.586991+0000","last_peered":"2024-09-22T03:34:26.586991+0000","last_clean":"2024-09-22T03:34:26.586991+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:34:26.586991+0000","last_undegraded":"2024-09-22T03:34:26.586991+0000","last_fullsized":"2024-09-22T03:34:26.586991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'231","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:25.275958+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:34:25.275958+0000","last_peered":"2024-09-22T03:34:25.275958+0000","last_clean":"2024-09-22T03:34:25.275958+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:34:25.275958+0000","last_undegraded":"2024-09-22T03:34:25.275958+0000","last_fullsized":"2024-09-22T03:34:25.275958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'207","reported_seq":220,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:25.985002+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:34:25.985002+0000","last_peered":"2024-09-22T03:34:25.985002+0000","last_clean":"2024-09-22T03:34:25.985002+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:34:25.985002+0000","last_undegraded":"2024-09-22T03:34:25.985002+0000","last_fullsized":"2024-09-22T03:34:25.985002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'227","reported_seq":240,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.994523+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:34:26.994523+0000","last_peered":"2024-09-22T03:34:26.994523+0000","last_clean":"2024-09-22T03:34:26.994523+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:34:26.994523+0000","last_undegraded":"2024-09-22T03:34:26.994523+0000","last_fullsized":"2024-09-22T03:34:26.994523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1867776,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":228,"num_write_kb":1824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'223","reported_seq":236,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.977826+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:34:26.977826+0000","last_peered":"2024-09-22T03:34:26.977826+0000","last_clean":"2024-09-22T03:34:26.977826+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:34:26.977826+0000","last_undegraded":"2024-09-22T03:34:26.977826+0000","last_fullsized":"2024-09-22T03:34:26.977826+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'215","reported_seq":228,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:25.673402+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:34:25.673402+0000","last_peered":"2024-09-22T03:34:25.673402+0000","last_clean":"2024-09-22T03:34:25.673402+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:34:25.673402+0000","last_undegraded":"2024-09-22T03:34:25.673402+0000","last_fullsized":"2024-09-22T03:34:25.673402+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'199","reported_seq":212,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:23.857788+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:34:23.857788+0000","last_peered":"2024-09-22T03:34:23.857788+0000","last_clean":"2024-09-22T03:34:23.857788+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:34:23.857788+0000","last_undegraded":"2024-09-22T03:34:23.857788+0000","last_fullsized":"2024-09-22T03:34:23.857788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'247","reported_seq":260,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.935716+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:34:26.935716+0000","last_peered":"2024-09-22T03:34:26.935716+0000","last_clean":"2024-09-22T03:34:26.935716+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:34:26.935716+0000","last_undegraded":"2024-09-22T03:34:26.935716+0000","last_fullsized":"2024-09-22T03:34:26.935716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'183","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.778933+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:34:26.778933+0000","last_peered":"2024-09-22T03:34:26.778933+0000","last_clean":"2024-09-22T03:34:26.778933+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:34:26.778933+0000","last_undegraded":"2024-09-22T03:34:26.778933+0000","last_fullsized":"2024-09-22T03:34:26.778933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'319","reported_seq":332,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.745177+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:34:26.745177+0000","last_peered":"2024-09-22T03:34:26.745177+0000","last_clean":"2024-09-22T03:34:26.745177+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:34:26.745177+0000","last_undegraded":"2024-09-22T03:34:26.745177+0000","last_fullsized":"2024-09-22T03:34:26.745177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'311","reported_seq":324,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.889767+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:34:26.889767+0000","last_peered":"2024-09-22T03:34:26.889767+0000","last_clean":"2024-09-22T03:34:26.889767+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:34:26.889767+0000","last_undegraded":"2024-09-22T03:34:26.889767+0000","last_fullsized":"2024-09-22T03:34:26.889767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"88'223","reported_seq":236,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.473796+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:34:26.473796+0000","last_peered":"2024-09-22T03:34:26.473796+0000","last_clean":"2024-09-22T03:34:26.473796+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:34:26.473796+0000","last_undegraded":"2024-09-22T03:34:26.473796+0000","last_fullsized":"2024-09-22T03:34:26.473796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'335","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:25.847898+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:34:25.847898+0000","last_peered":"2024-09-22T03:34:25.847898+0000","last_clean":"2024-09-22T03:34:25.847898+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:34:25.847898+0000","last_undegraded":"2024-09-22T03:34:25.847898+0000","last_fullsized":"2024-09-22T03:34:25.847898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'231","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.797030+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:34:26.797030+0000","last_peered":"2024-09-22T03:34:26.797030+0000","last_clean":"2024-09-22T03:34:26.797030+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:34:26.797030+0000","last_undegraded":"2024-09-22T03:34:26.797030+0000","last_fullsized":"2024-09-22T03:34:26.797030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":232,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'295","reported_seq":308,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:26.843482+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:34:26.843482+0000","last_peered":"2024-09-22T03:34:26.843482+0000","last_clean":"2024-09-22T03:34:26.843482+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:34:26.843482+0000","last_undegraded":"2024-09-22T03:34:26.843482+0000","last_fullsized":"2024-09-22T03:34:26.843482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"88'183","reported_seq":196,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:25.462827+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:34:25.462827+0000","last_peered":"2024-09-22T03:34:25.462827+0000","last_clean":"2024-09-22T03:34:25.462827+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:34:25.462827+0000","last_undegraded":"2024-09-22T03:34:25.462827+0000","last_fullsized":"2024-09-22T03:34:25.462827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":31686656,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3868,"num_write_kb":30944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214008320000,"internally_reserved":0,"allocated":203173888,"data_stored":203173888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3852,"ondisk_log_size":3852,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214008328192,"internally_reserved":0,"allocated":203165696,"data_stored":203165696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53497729024,"internally_reserved":0,"allocated":55144448,"data_stored":55144448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":52304,"kb_used_data":52304,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52245424,"statfs":{"total":53552873472,"available":53499314176,"internally_reserved":0,"allocated":53559296,"data_stored":53559296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":40672,"kb_used_data":40672,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52257056,"statfs":{"total":53552873472,"available":53511225344,"internally_reserved":0,"allocated":41648128,"data_stored":41648128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":39188,"kb_used_data":39188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52258540,"statfs":{"total":53552873472,"available":53512744960,"internally_reserved":0,"allocated":40128512,"data_stored":40128512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":59968,"kb_used_data":59968,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52237760,"statfs":{"total":53552873472,"available":53491466240,"internally_reserved":0,"allocated":61407232,"data_stored":61407232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53497729024,"internally_reserved":0,"allocated":55144448,"data_stored":55144448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53489373184,"internally_reserved":0,"allocated":63500288,"data_stored":63500288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53511360512,"internally_reserved":0,"allocated":41512960,"data_stored":41512960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53509873664,"internally_reserved":0,"allocated":42999808,"data_stored":42999808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53497720832,"internally_reserved":0,"allocated":55152640,"data_stored":55152640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53489373184,"internally_reserved":0,"allocated":63500288,"data_stored":63500288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53511360512,"internally_reserved":0,"allocated":41512960,"data_stored":41512960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53509865472,"internally_reserved":0,"allocated":43008000,"data_stored":43008000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53497720832,"internally_reserved":0,"allocated":55152640,"data_stored":55152640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:34:28.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 4051 4035 1.16743 1.17188 0.211809 0.10674 2024-09-22T03:34:28.869 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:29.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:29.022+0000 7fb249997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:29.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:29.030+0000 7fb249997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:29.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 4214 4198 1.17122 1.27344 0.0288339 0.106312 2024-09-22T03:34:30.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 4365 4349 1.17151 1.17969 0.205729 0.106485 2024-09-22T03:34:31.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:31.201 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 4507 4491 1.16943 1.10938 0.071956 0.106592 2024-09-22T03:34:31.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:31.309+0000 7f1eaeb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:31.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:31.315+0000 7f1eaeb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:32.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 4670 4654 1.17278 1.27344 0.082 0.106452 2024-09-22T03:34:33.127 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:33.127 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:33.127 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:34:33.127 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:34:33.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 16 4826 4810 1.17421 1.21875 0.106597 0.106241 2024-09-22T03:34:33.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.248+0000 7efc65b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.258+0000 7efc65b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:33.281 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:33.284 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:33.284 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:33.284 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:34:33.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.400+0000 7f9bbdb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.408+0000 7f9bbdb17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:33.430 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:33.430 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:33.431 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:33.434 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:33.434 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:33.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:34:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.526+0000 7f3fc9957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.531+0000 7f3fc9957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.607+0000 7f51500c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.617+0000 7f51500c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:33.679 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:33.680 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:33.680 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:33.680 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:33.683 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:33.683 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:33.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:34:33.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.837+0000 7f966a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:33.846+0000 7f966a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:33.874 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:33.874 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:34.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 4958 4942 1.16988 1.03125 0.0818733 0.106687 2024-09-22T03:34:35.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 5104 5088 1.16902 1.14062 0.137089 0.106678 2024-09-22T03:34:35.642 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:35.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:35.777+0000 7f644e6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:35.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:35.783+0000 7f644e6ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:36.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 5239 5223 1.16575 1.05469 0.0341342 0.106857 2024-09-22T03:34:37.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 5370 5354 1.16179 1.02344 0.200105 0.107443 2024-09-22T03:34:37.969 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:38.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:38.079+0000 7f67ded57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:38.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:38.084+0000 7f67ded57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:38.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 5491 5475 1.15594 0.945312 0.150748 0.107951 2024-09-22T03:34:38.875 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:38.875 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:38.876 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:34:38.876 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:34:39.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.020+0000 7fd4ebf4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.029+0000 7fd4ebf4e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.050 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:39.050 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:39.050 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:39.050 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:39.050 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:39.050 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:39.050 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:39.051 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:39.051 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:39.051 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:39.051 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:39.051 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:39.053 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:39.053 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:39.054 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:34:39.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 5636 5620 1.15533 1.13281 0.0792492 0.108069 2024-09-22T03:34:39.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.198+0000 7fe663197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.210+0000 7fe663197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.242 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:39.242 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:39.242 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:39.242 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:39.243 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:39.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:39.247 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:39.247 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:34:39.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.373+0000 7f2f9866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.386+0000 7f2f9866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.402 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:39.402 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:39.403 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:39.406 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:39.406 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:39.406 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:34:39.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.524+0000 7faf55d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:39.533+0000 7faf55d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:39.562 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:39.562 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:39.562 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:39.563 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:39.566 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:39.566 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:40.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 5771 5755 1.15274 1.05469 0.0448531 0.108227 2024-09-22T03:34:40.241 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:40.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:40.367+0000 7f367622c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:40.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:40.375+0000 7f367622c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:41.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:34:41.202391+0000 min lat: 0.0135566 max lat: 0.501871 avg lat: 0.108422 2024-09-22T03:34:41.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:34:41.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 5912 5896 1.15146 1.10156 0.0714551 0.108422 2024-09-22T03:34:42.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 6057 6041 1.151 1.13281 0.112789 0.108385 2024-09-22T03:34:42.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:42.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:42.614+0000 7f9740b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:42.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:42.620+0000 7f9740b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:43.202 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 6200 6184 1.1502 1.11719 0.138693 0.108508 2024-09-22T03:34:44.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 16 6348 6332 1.15033 1.15625 0.111484 0.108438 2024-09-22T03:34:44.567 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:44.567 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:44.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:34:44.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:44.719+0000 7f099ae2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:44.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:44.728+0000 7f099ae2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:44.811 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:44.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:44.959+0000 7f1b4d317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:44.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:44.965+0000 7f1b4d317640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:45.042 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:34:45.042 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:34:45.064 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-22T03:34:44.065986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":49979955,"num_objects":758,"num_object_clones":0,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6168,"num_write_kb":50128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6113,"ondisk_log_size":6113,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":301896,"kb_used_data":301896,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208889016,"statfs":{"total":214211493888,"available":213902352384,"internally_reserved":0,"allocated":309141504,"data_stored":309141504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11444224,"num_objects":175,"num_object_clones":0,"num_object_copies":525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1397,"num_write_kb":11176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002419"},"pg_stats":[{"pgid":"9.7","version":"88'319","reported_seq":332,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.163508+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:34:41.163508+0000","last_peered":"2024-09-22T03:34:41.163508+0000","last_clean":"2024-09-22T03:34:41.163508+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:34:41.163508+0000","last_undegraded":"2024-09-22T03:34:41.163508+0000","last_fullsized":"2024-09-22T03:34:41.163508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'335","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:39.782616+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:34:39.782616+0000","last_peered":"2024-09-22T03:34:39.782616+0000","last_clean":"2024-09-22T03:34:39.782616+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:34:39.782616+0000","last_undegraded":"2024-09-22T03:34:39.782616+0000","last_fullsized":"2024-09-22T03:34:39.782616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'375","reported_seq":388,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.372442+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:34:41.372442+0000","last_peered":"2024-09-22T03:34:41.372442+0000","last_clean":"2024-09-22T03:34:41.372442+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:34:41.372442+0000","last_undegraded":"2024-09-22T03:34:41.372442+0000","last_fullsized":"2024-09-22T03:34:41.372442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'359","reported_seq":372,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.097695+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:34:41.097695+0000","last_peered":"2024-09-22T03:34:41.097695+0000","last_clean":"2024-09-22T03:34:41.097695+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:34:41.097695+0000","last_undegraded":"2024-09-22T03:34:41.097695+0000","last_fullsized":"2024-09-22T03:34:41.097695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2949120,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":2880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'383","reported_seq":396,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.737161+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:34:41.737161+0000","last_peered":"2024-09-22T03:34:41.737161+0000","last_clean":"2024-09-22T03:34:41.737161+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:34:41.737161+0000","last_undegraded":"2024-09-22T03:34:41.737161+0000","last_fullsized":"2024-09-22T03:34:41.737161+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'319","reported_seq":332,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.692908+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:34:41.692908+0000","last_peered":"2024-09-22T03:34:41.692908+0000","last_clean":"2024-09-22T03:34:41.692908+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:34:41.692908+0000","last_undegraded":"2024-09-22T03:34:41.692908+0000","last_fullsized":"2024-09-22T03:34:41.692908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'383","reported_seq":396,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.962026+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:34:41.962026+0000","last_peered":"2024-09-22T03:34:41.962026+0000","last_clean":"2024-09-22T03:34:41.962026+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:34:41.962026+0000","last_undegraded":"2024-09-22T03:34:41.962026+0000","last_fullsized":"2024-09-22T03:34:41.962026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'423","reported_seq":436,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.916340+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:34:41.916340+0000","last_peered":"2024-09-22T03:34:41.916340+0000","last_clean":"2024-09-22T03:34:41.916340+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:34:41.916340+0000","last_undegraded":"2024-09-22T03:34:41.916340+0000","last_fullsized":"2024-09-22T03:34:41.916340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'287","reported_seq":300,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.055646+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:34:41.055646+0000","last_peered":"2024-09-22T03:34:41.055646+0000","last_clean":"2024-09-22T03:34:41.055646+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:34:41.055646+0000","last_undegraded":"2024-09-22T03:34:41.055646+0000","last_fullsized":"2024-09-22T03:34:41.055646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'423","reported_seq":436,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:39.401054+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:34:39.401054+0000","last_peered":"2024-09-22T03:34:39.401054+0000","last_clean":"2024-09-22T03:34:39.401054+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:34:39.401054+0000","last_undegraded":"2024-09-22T03:34:39.401054+0000","last_fullsized":"2024-09-22T03:34:39.401054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'479","reported_seq":492,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:40.834028+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:34:40.834028+0000","last_peered":"2024-09-22T03:34:40.834028+0000","last_clean":"2024-09-22T03:34:40.834028+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:34:40.834028+0000","last_undegraded":"2024-09-22T03:34:40.834028+0000","last_fullsized":"2024-09-22T03:34:40.834028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"88'351","reported_seq":364,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.552236+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:34:41.552236+0000","last_peered":"2024-09-22T03:34:41.552236+0000","last_clean":"2024-09-22T03:34:41.552236+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:34:41.552236+0000","last_undegraded":"2024-09-22T03:34:41.552236+0000","last_fullsized":"2024-09-22T03:34:41.552236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'423","reported_seq":436,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.257414+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:34:41.257414+0000","last_peered":"2024-09-22T03:34:41.257414+0000","last_clean":"2024-09-22T03:34:41.257414+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:34:41.257414+0000","last_undegraded":"2024-09-22T03:34:41.257414+0000","last_fullsized":"2024-09-22T03:34:41.257414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":424,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'364","reported_seq":377,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.985581+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:34:41.985581+0000","last_peered":"2024-09-22T03:34:41.985581+0000","last_clean":"2024-09-22T03:34:41.985581+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:34:41.985581+0000","last_undegraded":"2024-09-22T03:34:41.985581+0000","last_fullsized":"2024-09-22T03:34:41.985581+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2990080,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":365,"num_write_kb":2920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'455","reported_seq":468,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.855483+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:34:41.855483+0000","last_peered":"2024-09-22T03:34:41.855483+0000","last_clean":"2024-09-22T03:34:41.855483+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:34:41.855483+0000","last_undegraded":"2024-09-22T03:34:41.855483+0000","last_fullsized":"2024-09-22T03:34:41.855483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"88'335","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:41.842214+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:34:41.842214+0000","last_peered":"2024-09-22T03:34:41.842214+0000","last_clean":"2024-09-22T03:34:41.842214+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:34:41.842214+0000","last_undegraded":"2024-09-22T03:34:41.842214+0000","last_fullsized":"2024-09-22T03:34:41.842214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":49389568,"num_objects":754,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6029,"num_write_kb":48232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213896630272,"internally_reserved":0,"allocated":314863616,"data_stored":314863616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6013,"ondisk_log_size":6013,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213896638464,"internally_reserved":0,"allocated":314855424,"data_stored":314855424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53467607040,"internally_reserved":0,"allocated":85266432,"data_stored":85266432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706363,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":81728,"kb_used_data":81728,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52216000,"statfs":{"total":53552873472,"available":53469184000,"internally_reserved":0,"allocated":83689472,"data_stored":83689472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706363,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":65028,"kb_used_data":65028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52232700,"statfs":{"total":53552873472,"available":53486284800,"internally_reserved":0,"allocated":66588672,"data_stored":66588672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706363,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":60312,"kb_used_data":60312,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52237416,"statfs":{"total":53552873472,"available":53491113984,"internally_reserved":0,"allocated":61759488,"data_stored":61759488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706363,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":94828,"kb_used_data":94828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52202900,"statfs":{"total":53552873472,"available":53455769600,"internally_reserved":0,"allocated":97103872,"data_stored":97103872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53467607040,"internally_reserved":0,"allocated":85266432,"data_stored":85266432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53453914112,"internally_reserved":0,"allocated":98959360,"data_stored":98959360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53490135040,"internally_reserved":0,"allocated":62738432,"data_stored":62738432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53484982272,"internally_reserved":0,"allocated":67891200,"data_stored":67891200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53467607040,"internally_reserved":0,"allocated":85266432,"data_stored":85266432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53453914112,"internally_reserved":0,"allocated":98959360,"data_stored":98959360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53490135040,"internally_reserved":0,"allocated":62738432,"data_stored":62738432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53484982272,"internally_reserved":0,"allocated":67891200,"data_stored":67891200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53467598848,"internally_reserved":0,"allocated":85274624,"data_stored":85274624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:34:45.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 6493 6477 1.14993 1.13281 0.0637765 0.108544 2024-09-22T03:34:46.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 6630 6614 1.14816 1.07031 0.151813 0.108704 2024-09-22T03:34:47.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:47.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 6791 6775 1.15054 1.25781 0.049914 0.108491 2024-09-22T03:34:47.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:47.213+0000 7efd7cd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:47.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:47.221+0000 7efd7cd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:48.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 16 6933 6917 1.14967 1.10938 0.138652 0.10855 2024-09-22T03:34:49.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 7105 7089 1.15371 1.34375 0.0594533 0.108244 2024-09-22T03:34:49.450 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:49.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:49.570+0000 7f884962c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:49.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:49.575+0000 7f884962c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.066 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:50.066 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:50.066 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:34:50.069 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:34:50.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.177+0000 7f8b0c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.185+0000 7f8b0c597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 7240 7224 1.15168 1.05469 0.39147 0.108374 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:50.208 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:50.210 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:50.210 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:50.210 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:34:50.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.326+0000 7fab68e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.335+0000 7fab68e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:50.352 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:50.355 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:50.355 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:50.356 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:34:50.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.470+0000 7f349d917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.481+0000 7f349d917640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.496 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:50.496 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:50.496 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:50.496 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:50.497 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:50.500 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:50.500 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:50.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:34:50.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.630+0000 7fd67602c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:50.642+0000 7fd67602c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:50.661 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:50.661 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:50.662 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:50.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:50.665 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:51.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 7388 7372 1.15177 1.15625 0.0440586 0.108442 2024-09-22T03:34:51.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:51.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:51.890+0000 7f7b644d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:51.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:51.896+0000 7f7b644d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:52.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 7529 7513 1.15079 1.10156 0.0396763 0.108403 2024-09-22T03:34:53.203 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 7694 7678 1.15344 1.28906 0.090367 0.108277 2024-09-22T03:34:54.123 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:54.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 7835 7819 1.15246 1.10156 0.168373 0.108326 2024-09-22T03:34:54.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:54.238+0000 7f72f0a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:54.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:54.244+0000 7f72f0a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:55.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 7956 7940 1.14862 0.945312 0.12063 0.108683 2024-09-22T03:34:55.666 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:34:55.666 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:34:55.666 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:34:55.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:34:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:55.780+0000 7fe60426c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:55.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:55.790+0000 7fe60426c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:55.826 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:55.826 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:55.826 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:55.826 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:55.826 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:55.826 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:55.826 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:55.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:55.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:55.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:55.827 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:55.827 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:55.829 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:55.830 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:55.830 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:34:55.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:55.940+0000 7f87d9157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:55.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:55.949+0000 7f87d9157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:56.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:56.030 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:56.030 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:56.033 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:56.034 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:56.034 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:34:56.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:56.144+0000 7fbe8b517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:56.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:56.152+0000 7fbe8b517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:56.170 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:56.170 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:56.171 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:56.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:56.174 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:56.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:34:56.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 8074 8058 1.1445 0.921875 0.0794154 0.10908 2024-09-22T03:34:56.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:56.318+0000 7f9dfa597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:56.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:56.333+0000 7f9dfa597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:56.356 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:34:56.356 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:34:56.356 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:34:56.356 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:34:56.356 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:34:56.357 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:34:56.357 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:34:56.357 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:34:56.357 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:34:56.357 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:34:56.357 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:34:56.357 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:34:56.359 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:34:56.360 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:34:56.436 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:34:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:56.548+0000 7ff291b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:56.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:56.554+0000 7ff291b97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:57.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 8201 8185 1.14178 0.992188 0.185949 0.10929 2024-09-22T03:34:58.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 8346 8330 1.14162 1.13281 0.113311 0.109397 2024-09-22T03:34:58.680 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:34:58.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:58.826+0000 7ff360717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:58.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:34:58.831+0000 7ff360717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:34:59.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 8482 8466 1.14025 1.0625 0.0529276 0.109396 2024-09-22T03:35:00.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 8625 8609 1.13986 1.11719 0.129242 0.109507 2024-09-22T03:35:00.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:01.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:01.085+0000 7eff43d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:01.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:01.091+0000 7eff43d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:01.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:35:01.204218+0000 min lat: 0.0135566 max lat: 0.501871 avg lat: 0.109651 2024-09-22T03:35:01.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:35:01.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 8761 8745 1.13857 1.0625 0.164811 0.109651 2024-09-22T03:35:01.360 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:01.360 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:01.361 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:01.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:01.514+0000 7fad6bd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:01.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:01.525+0000 7fad6bd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:01.875 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:01.876 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:01.895 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-22T03:35:00.068842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67551795,"num_objects":1026,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8313,"num_write_kb":67288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8258,"ondisk_log_size":8258,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":410072,"kb_used_data":410072,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208780840,"statfs":{"total":214211493888,"available":213791580160,"internally_reserved":0,"allocated":419913728,"data_stored":419913728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11640832,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1421,"num_write_kb":11368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002162"},"pg_stats":[{"pgid":"9.7","version":"88'495","reported_seq":508,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.949687+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:34:56.949687+0000","last_peered":"2024-09-22T03:34:56.949687+0000","last_clean":"2024-09-22T03:34:56.949687+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:34:56.949687+0000","last_undegraded":"2024-09-22T03:34:56.949687+0000","last_fullsized":"2024-09-22T03:34:56.949687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'463","reported_seq":476,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.743072+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:34:56.743072+0000","last_peered":"2024-09-22T03:34:56.743072+0000","last_clean":"2024-09-22T03:34:56.743072+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:34:56.743072+0000","last_undegraded":"2024-09-22T03:34:56.743072+0000","last_fullsized":"2024-09-22T03:34:56.743072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'511","reported_seq":524,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:55.938084+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:34:55.938084+0000","last_peered":"2024-09-22T03:34:55.938084+0000","last_clean":"2024-09-22T03:34:55.938084+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:34:55.938084+0000","last_undegraded":"2024-09-22T03:34:55.938084+0000","last_fullsized":"2024-09-22T03:34:55.938084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'447","reported_seq":460,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.849504+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:34:56.849504+0000","last_peered":"2024-09-22T03:34:56.849504+0000","last_clean":"2024-09-22T03:34:56.849504+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:34:56.849504+0000","last_undegraded":"2024-09-22T03:34:56.849504+0000","last_fullsized":"2024-09-22T03:34:56.849504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'503","reported_seq":516,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.066389+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:34:56.066389+0000","last_peered":"2024-09-22T03:34:56.066389+0000","last_clean":"2024-09-22T03:34:56.066389+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:34:56.066389+0000","last_undegraded":"2024-09-22T03:34:56.066389+0000","last_fullsized":"2024-09-22T03:34:56.066389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'455","reported_seq":468,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.014539+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:34:56.014539+0000","last_peered":"2024-09-22T03:34:56.014539+0000","last_clean":"2024-09-22T03:34:56.014539+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:34:56.014539+0000","last_undegraded":"2024-09-22T03:34:56.014539+0000","last_fullsized":"2024-09-22T03:34:56.014539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'551","reported_seq":564,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.788931+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:34:56.788931+0000","last_peered":"2024-09-22T03:34:56.788931+0000","last_clean":"2024-09-22T03:34:56.788931+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:34:56.788931+0000","last_undegraded":"2024-09-22T03:34:56.788931+0000","last_fullsized":"2024-09-22T03:34:56.788931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'597","reported_seq":610,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.992290+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:34:56.992290+0000","last_peered":"2024-09-22T03:34:56.992290+0000","last_clean":"2024-09-22T03:34:56.992290+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:34:56.992290+0000","last_undegraded":"2024-09-22T03:34:56.992290+0000","last_fullsized":"2024-09-22T03:34:56.992290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4898816,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":598,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'415","reported_seq":428,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.643595+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:34:56.643595+0000","last_peered":"2024-09-22T03:34:56.643595+0000","last_clean":"2024-09-22T03:34:56.643595+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:34:56.643595+0000","last_undegraded":"2024-09-22T03:34:56.643595+0000","last_fullsized":"2024-09-22T03:34:56.643595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'559","reported_seq":572,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.913979+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:34:56.913979+0000","last_peered":"2024-09-22T03:34:56.913979+0000","last_clean":"2024-09-22T03:34:56.913979+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:34:56.913979+0000","last_undegraded":"2024-09-22T03:34:56.913979+0000","last_fullsized":"2024-09-22T03:34:56.913979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4587520,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'599","reported_seq":612,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:55.699431+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:34:55.699431+0000","last_peered":"2024-09-22T03:34:55.699431+0000","last_clean":"2024-09-22T03:34:55.699431+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:34:55.699431+0000","last_undegraded":"2024-09-22T03:34:55.699431+0000","last_fullsized":"2024-09-22T03:34:55.699431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":4800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"88'495","reported_seq":508,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.211644+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:34:56.211644+0000","last_peered":"2024-09-22T03:34:56.211644+0000","last_clean":"2024-09-22T03:34:56.211644+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:34:56.211644+0000","last_undegraded":"2024-09-22T03:34:56.211644+0000","last_fullsized":"2024-09-22T03:34:56.211644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'567","reported_seq":580,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:55.522605+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:34:55.522605+0000","last_peered":"2024-09-22T03:34:55.522605+0000","last_clean":"2024-09-22T03:34:55.522605+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:34:55.522605+0000","last_undegraded":"2024-09-22T03:34:55.522605+0000","last_fullsized":"2024-09-22T03:34:55.522605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'527","reported_seq":540,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.364681+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:34:56.364681+0000","last_peered":"2024-09-22T03:34:56.364681+0000","last_clean":"2024-09-22T03:34:56.364681+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:34:56.364681+0000","last_undegraded":"2024-09-22T03:34:56.364681+0000","last_fullsized":"2024-09-22T03:34:56.364681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'567","reported_seq":580,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.192648+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:34:56.192648+0000","last_peered":"2024-09-22T03:34:56.192648+0000","last_clean":"2024-09-22T03:34:56.192648+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:34:56.192648+0000","last_undegraded":"2024-09-22T03:34:56.192648+0000","last_fullsized":"2024-09-22T03:34:56.192648+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"88'407","reported_seq":420,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:56.476439+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:34:56.476439+0000","last_peered":"2024-09-22T03:34:56.476439+0000","last_clean":"2024-09-22T03:34:56.476439+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:34:56.476439+0000","last_undegraded":"2024-09-22T03:34:56.476439+0000","last_fullsized":"2024-09-22T03:34:56.476439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":66961408,"num_objects":1022,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8174,"num_write_kb":65392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213786730496,"internally_reserved":0,"allocated":424763392,"data_stored":424763392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8158,"ondisk_log_size":8158,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213786738688,"internally_reserved":0,"allocated":424755200,"data_stored":424755200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53438832640,"internally_reserved":0,"allocated":114040832,"data_stored":114040832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":110008,"kb_used_data":110008,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52187720,"statfs":{"total":53552873472,"available":53440225280,"internally_reserved":0,"allocated":112648192,"data_stored":112648192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":87516,"kb_used_data":87516,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52210212,"statfs":{"total":53552873472,"available":53463257088,"internally_reserved":0,"allocated":89616384,"data_stored":89616384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":83632,"kb_used_data":83632,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52214096,"statfs":{"total":53552873472,"available":53467234304,"internally_reserved":0,"allocated":85639168,"data_stored":85639168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":128916,"kb_used_data":128916,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52168812,"statfs":{"total":53552873472,"available":53420863488,"internally_reserved":0,"allocated":132009984,"data_stored":132009984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53438832640,"internally_reserved":0,"allocated":114040832,"data_stored":114040832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53419540480,"internally_reserved":0,"allocated":133332992,"data_stored":133332992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53465829376,"internally_reserved":0,"allocated":87044096,"data_stored":87044096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53462544384,"internally_reserved":0,"allocated":90329088,"data_stored":90329088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53438824448,"internally_reserved":0,"allocated":114049024,"data_stored":114049024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53419532288,"internally_reserved":0,"allocated":133341184,"data_stored":133341184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53465829376,"internally_reserved":0,"allocated":87044096,"data_stored":87044096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53462544384,"internally_reserved":0,"allocated":90329088,"data_stored":90329088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53438824448,"internally_reserved":0,"allocated":114049024,"data_stored":114049024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:02.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 8881 8865 1.13527 0.9375 0.0536231 0.109816 2024-09-22T03:35:03.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 9009 8993 1.13309 1 0.299422 0.11008 2024-09-22T03:35:03.272 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:03.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:03.381+0000 7fc96a6a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:03.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:03.387+0000 7fc96a6a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:04.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 9159 9143 1.1337 1.17188 0.069409 0.110166 2024-09-22T03:35:05.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 16 9287 9271 1.13161 1 0.0781845 0.110353 2024-09-22T03:35:05.556 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:05.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:05.663+0000 7ff495ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:05.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:05.668+0000 7ff495ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:06.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 9410 9394 1.12899 0.960938 0.090764 0.110623 2024-09-22T03:35:06.897 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:06.898 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:06.898 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:35:06.898 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:35:07.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.004+0000 7f2b7210e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.013+0000 7f2b7210e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:07.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:07.031 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:07.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:35:07.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.144+0000 7fee8730e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.153+0000 7fee8730e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.202 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:07.202 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:07.202 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:07.202 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:07.203 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:07.204 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 9539 9523 1.12715 1.00781 0.113296 0.110789 2024-09-22T03:35:07.206 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:07.207 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:07.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:35:07.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.359+0000 7f65a6cfa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.370+0000 7f65a6cfa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:07.434 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:07.436 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:07.437 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:07.437 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:35:07.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.553+0000 7fabfd157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.562+0000 7fabfd157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:07.584 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:07.584 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:07.584 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:07.584 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:07.584 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:07.584 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:07.587 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:07.587 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:07.861 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:07.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.986+0000 7f1e16c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:07.994+0000 7f1e16c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:08.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 9671 9655 1.12572 1.03125 0.056349 0.1109 2024-09-22T03:35:09.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 9806 9790 1.12467 1.05469 0.132267 0.111024 2024-09-22T03:35:10.160 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:10.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 9955 9939 1.12524 1.16406 0.0901227 0.111008 2024-09-22T03:35:10.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:10.291+0000 7f2d824a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:10.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:10.299+0000 7f2d824a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:11.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 10107 10091 1.12613 1.1875 0.0981893 0.110925 2024-09-22T03:35:12.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 10261 10245 1.12721 1.20312 0.0605531 0.11081 2024-09-22T03:35:12.455 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:12.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:12.573+0000 7f51a9c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:12.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:12.578+0000 7f51a9c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:12.588 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:12.588 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:12.588 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:35:12.588 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:35:12.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:12.711+0000 7fb339397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:12.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:12.726+0000 7fb339397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:12.749 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:12.752 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:12.753 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:12.753 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:35:12.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:12.930+0000 7f136bc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:12.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:12.945+0000 7f136bc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:13.045 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:13.046 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:13.049 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:13.050 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:13.050 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:35:13.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:13.172+0000 7f93736ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:13.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:13.185+0000 7f93736ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:13.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 10412 10396 1.12794 1.17969 0.155422 0.110769 2024-09-22T03:35:13.249 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:13.249 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:13.250 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:13.253 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:13.253 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:13.253 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:35:13.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:13.408+0000 7fea9522c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:13.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:13.419+0000 7fea9522c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:13.445 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:13.445 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:13.445 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:13.445 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:13.445 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:13.445 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:13.445 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:13.446 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:13.446 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:13.446 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:13.446 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:13.446 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:13.448 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:13.448 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:14.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 10562 10546 1.12854 1.17188 0.115039 0.110678 2024-09-22T03:35:14.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:14.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:14.845+0000 7fe10a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:14.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:14.851+0000 7fe10a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:15.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 10703 10687 1.12817 1.10156 0.187016 0.110679 2024-09-22T03:35:16.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 10858 10842 1.12927 1.21094 0.0630462 0.110545 2024-09-22T03:35:17.036 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:17.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:17.191+0000 7f591324f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:17.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:17.197+0000 7f591324f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:17.205 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 10985 10969 1.12747 0.992188 0.196091 0.110709 2024-09-22T03:35:18.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 11133 11117 1.12784 1.15625 0.0296609 0.110707 2024-09-22T03:35:18.450 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:18.450 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:18.450 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:35:18.450 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:35:18.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:18.569+0000 7f2cc4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:18.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:18.579+0000 7f2cc4597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:18.613 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:18.617 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:18.617 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:18.617 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:35:18.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:18.772+0000 7fe79b24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:18.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:18.787+0000 7fe79b24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:18.872 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:18.873 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:18.876 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:18.876 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:18.877 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:35:19.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:19.017+0000 7fa4e6ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:19.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:19.030+0000 7fa4e6ed7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:19.057 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:19.057 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:19.057 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:19.057 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:19.058 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:19.061 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:19.061 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:19.062 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:35:19.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:19.174+0000 7f9ef726c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:19.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:19.183+0000 7f9ef726c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:19.200 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:19.201 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:19.203 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:19.203 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:19.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 16 11273 11257 1.1274 1.09375 0.178773 0.110731 2024-09-22T03:35:19.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:19.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:19.496+0000 7fefb1b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:19.503+0000 7fefb1b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:20.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 11405 11389 1.12618 1.03125 0.137456 0.110897 2024-09-22T03:35:21.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:35:21.206026+0000 min lat: 0.0135566 max lat: 0.501871 avg lat: 0.111034 2024-09-22T03:35:21.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:35:21.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 11531 11515 1.12441 0.984375 0.316056 0.111034 2024-09-22T03:35:21.636 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:21.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:21.757+0000 7f5dcbc2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:21.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:21.762+0000 7f5dcbc2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:22.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 11676 11660 1.12451 1.13281 0.0796539 0.111057 2024-09-22T03:35:23.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 11827 11811 1.12518 1.17969 0.0981761 0.111029 2024-09-22T03:35:23.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:24.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:24.078+0000 7fae6b597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:24.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:24.085+0000 7fae6b597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:24.204 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:24.204 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:24.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:24.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 11974 11958 1.12546 1.14844 0.130378 0.110945 2024-09-22T03:35:24.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:24.353+0000 7ff0ad157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:24.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:24.362+0000 7ff0ad157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:24.716 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:24.717 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:24.735 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-22T03:35:24.073031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95961651,"num_objects":1460,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11781,"num_write_kb":95032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11726,"ondisk_log_size":11726,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":582920,"kb_used_data":582920,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208607992,"statfs":{"total":214211493888,"available":213614583808,"internally_reserved":0,"allocated":596910080,"data_stored":596910080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11583488,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1414,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002216"},"pg_stats":[{"pgid":"9.7","version":"88'751","reported_seq":764,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.845485+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:35:21.845485+0000","last_peered":"2024-09-22T03:35:21.845485+0000","last_clean":"2024-09-22T03:35:21.845485+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:35:21.845485+0000","last_undegraded":"2024-09-22T03:35:21.845485+0000","last_fullsized":"2024-09-22T03:35:21.845485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"88'727","reported_seq":740,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.421987+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:35:21.421987+0000","last_peered":"2024-09-22T03:35:21.421987+0000","last_clean":"2024-09-22T03:35:21.421987+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:35:21.421987+0000","last_undegraded":"2024-09-22T03:35:21.421987+0000","last_fullsized":"2024-09-22T03:35:21.421987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'663","reported_seq":676,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:16.725900+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:35:16.725900+0000","last_peered":"2024-09-22T03:35:16.725900+0000","last_clean":"2024-09-22T03:35:16.725900+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:35:16.725900+0000","last_undegraded":"2024-09-22T03:35:16.725900+0000","last_fullsized":"2024-09-22T03:35:16.725900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"88'663","reported_seq":676,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:20.254855+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:35:20.254855+0000","last_peered":"2024-09-22T03:35:20.254855+0000","last_clean":"2024-09-22T03:35:20.254855+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:35:20.254855+0000","last_undegraded":"2024-09-22T03:35:20.254855+0000","last_fullsized":"2024-09-22T03:35:20.254855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'799","reported_seq":812,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.455933+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:35:21.455933+0000","last_peered":"2024-09-22T03:35:21.455933+0000","last_clean":"2024-09-22T03:35:21.455933+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:35:21.455933+0000","last_undegraded":"2024-09-22T03:35:21.455933+0000","last_fullsized":"2024-09-22T03:35:21.455933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553600,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":800,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'679","reported_seq":692,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.653633+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:35:21.653633+0000","last_peered":"2024-09-22T03:35:21.653633+0000","last_clean":"2024-09-22T03:35:21.653633+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:35:21.653633+0000","last_undegraded":"2024-09-22T03:35:21.653633+0000","last_fullsized":"2024-09-22T03:35:21.653633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'687","reported_seq":700,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.168685+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:35:21.168685+0000","last_peered":"2024-09-22T03:35:21.168685+0000","last_clean":"2024-09-22T03:35:21.168685+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:35:21.168685+0000","last_undegraded":"2024-09-22T03:35:21.168685+0000","last_fullsized":"2024-09-22T03:35:21.168685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"88'775","reported_seq":788,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:17.309578+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:35:17.309578+0000","last_peered":"2024-09-22T03:35:17.309578+0000","last_clean":"2024-09-22T03:35:17.309578+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:35:17.309578+0000","last_undegraded":"2024-09-22T03:35:17.309578+0000","last_fullsized":"2024-09-22T03:35:17.309578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'703","reported_seq":716,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.893978+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:35:21.893978+0000","last_peered":"2024-09-22T03:35:21.893978+0000","last_clean":"2024-09-22T03:35:21.893978+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:35:21.893978+0000","last_undegraded":"2024-09-22T03:35:21.893978+0000","last_fullsized":"2024-09-22T03:35:21.893978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":367,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141178+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:34:01.141178+0000","last_peered":"2024-09-22T03:34:01.141178+0000","last_clean":"2024-09-22T03:34:01.141178+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:34:01.141178+0000","last_undegraded":"2024-09-22T03:34:01.141178+0000","last_fullsized":"2024-09-22T03:34:01.141178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":233,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.141850+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:34:01.141850+0000","last_peered":"2024-09-22T03:34:01.141850+0000","last_clean":"2024-09-22T03:34:01.141850+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:34:01.141850+0000","last_undegraded":"2024-09-22T03:34:01.141850+0000","last_fullsized":"2024-09-22T03:34:01.141850+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'791","reported_seq":804,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.379924+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:35:21.379924+0000","last_peered":"2024-09-22T03:35:21.379924+0000","last_clean":"2024-09-22T03:35:21.379924+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:35:21.379924+0000","last_undegraded":"2024-09-22T03:35:21.379924+0000","last_fullsized":"2024-09-22T03:35:21.379924+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.143507+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:34:01.143507+0000","last_peered":"2024-09-22T03:34:01.143507+0000","last_clean":"2024-09-22T03:34:01.143507+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:34:01.143507+0000","last_undegraded":"2024-09-22T03:34:01.143507+0000","last_fullsized":"2024-09-22T03:34:01.143507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'831","reported_seq":844,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.955224+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:35:21.955224+0000","last_peered":"2024-09-22T03:35:21.955224+0000","last_clean":"2024-09-22T03:35:21.955224+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:35:21.955224+0000","last_undegraded":"2024-09-22T03:35:21.955224+0000","last_fullsized":"2024-09-22T03:35:21.955224+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.139087+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.139087+0000","last_peered":"2024-09-22T03:34:01.139087+0000","last_clean":"2024-09-22T03:34:01.139087+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:34:01.139087+0000","last_undegraded":"2024-09-22T03:34:01.139087+0000","last_fullsized":"2024-09-22T03:34:01.139087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"88'687","reported_seq":700,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.788982+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:35:21.788982+0000","last_peered":"2024-09-22T03:35:21.788982+0000","last_clean":"2024-09-22T03:35:21.788982+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:35:21.788982+0000","last_undegraded":"2024-09-22T03:35:21.788982+0000","last_fullsized":"2024-09-22T03:35:21.788982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140785+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:34:01.140785+0000","last_peered":"2024-09-22T03:34:01.140785+0000","last_clean":"2024-09-22T03:34:01.140785+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:34:01.140785+0000","last_undegraded":"2024-09-22T03:34:01.140785+0000","last_fullsized":"2024-09-22T03:34:01.140785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'791","reported_seq":804,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:19.703325+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:35:19.703325+0000","last_peered":"2024-09-22T03:35:19.703325+0000","last_clean":"2024-09-22T03:35:19.703325+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:35:19.703325+0000","last_undegraded":"2024-09-22T03:35:19.703325+0000","last_fullsized":"2024-09-22T03:35:19.703325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.147483+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.147483+0000","last_peered":"2024-09-22T03:34:01.147483+0000","last_clean":"2024-09-22T03:34:01.147483+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:34:01.147483+0000","last_undegraded":"2024-09-22T03:34:01.147483+0000","last_fullsized":"2024-09-22T03:34:01.147483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'743","reported_seq":756,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:19.754631+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:35:19.754631+0000","last_peered":"2024-09-22T03:35:19.754631+0000","last_clean":"2024-09-22T03:35:19.754631+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:35:19.754631+0000","last_undegraded":"2024-09-22T03:35:19.754631+0000","last_fullsized":"2024-09-22T03:35:19.754631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.145414+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:34:01.145414+0000","last_peered":"2024-09-22T03:34:01.145414+0000","last_clean":"2024-09-22T03:34:01.145414+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:34:01.145414+0000","last_undegraded":"2024-09-22T03:34:01.145414+0000","last_fullsized":"2024-09-22T03:34:01.145414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'767","reported_seq":780,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.083646+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:35:21.083646+0000","last_peered":"2024-09-22T03:35:21.083646+0000","last_clean":"2024-09-22T03:35:21.083646+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:35:21.083646+0000","last_undegraded":"2024-09-22T03:35:21.083646+0000","last_fullsized":"2024-09-22T03:35:21.083646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.146064+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:34:01.146064+0000","last_peered":"2024-09-22T03:34:01.146064+0000","last_clean":"2024-09-22T03:34:01.146064+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:34:01.146064+0000","last_undegraded":"2024-09-22T03:34:01.146064+0000","last_fullsized":"2024-09-22T03:34:01.146064+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"88'569","reported_seq":582,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:35:21.979934+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:35:21.979934+0000","last_peered":"2024-09-22T03:35:21.979934+0000","last_clean":"2024-09-22T03:35:21.979934+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:35:21.979934+0000","last_undegraded":"2024-09-22T03:35:21.979934+0000","last_fullsized":"2024-09-22T03:35:21.979934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4669440,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":570,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-22T03:34:01.140662+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:34:01.140662+0000","last_peered":"2024-09-22T03:34:01.140662+0000","last_clean":"2024-09-22T03:34:01.140662+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:34:01.140662+0000","last_undegraded":"2024-09-22T03:34:01.140662+0000","last_fullsized":"2024-09-22T03:34:01.140662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":95371264,"num_objects":1456,"num_object_clones":0,"num_object_copies":4368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11642,"num_write_kb":93136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213609037824,"internally_reserved":0,"allocated":602456064,"data_stored":602456064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11626,"ondisk_log_size":11626,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213609119744,"internally_reserved":0,"allocated":602374144,"data_stored":602374144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53390213120,"internally_reserved":0,"allocated":162660352,"data_stored":162660352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706403,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":157088,"kb_used_data":157088,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52140640,"statfs":{"total":53552873472,"available":53392015360,"internally_reserved":0,"allocated":160858112,"data_stored":160858112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706403,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":123584,"kb_used_data":123584,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52174144,"statfs":{"total":53552873472,"available":53426323456,"internally_reserved":0,"allocated":126550016,"data_stored":126550016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706403,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":120248,"kb_used_data":120248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52177480,"statfs":{"total":53552873472,"available":53429739520,"internally_reserved":0,"allocated":123133952,"data_stored":123133952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706403,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":182000,"kb_used_data":182000,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52115728,"statfs":{"total":53552873472,"available":53366505472,"internally_reserved":0,"allocated":186368000,"data_stored":186368000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53390213120,"internally_reserved":0,"allocated":162660352,"data_stored":162660352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53364924416,"internally_reserved":0,"allocated":187949056,"data_stored":187949056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53428584448,"internally_reserved":0,"allocated":124289024,"data_stored":124289024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53425397760,"internally_reserved":0,"allocated":127475712,"data_stored":127475712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53390213120,"internally_reserved":0,"allocated":162660352,"data_stored":162660352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53364850688,"internally_reserved":0,"allocated":188022784,"data_stored":188022784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53428576256,"internally_reserved":0,"allocated":124297216,"data_stored":124297216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53425397760,"internally_reserved":0,"allocated":127475712,"data_stored":127475712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53390213120,"internally_reserved":0,"allocated":162660352,"data_stored":162660352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:25.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 12118 12102 1.12546 1.125 0.127163 0.110968 2024-09-22T03:35:26.206 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 12265 12249 1.12573 1.14844 0.184731 0.110939 2024-09-22T03:35:26.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:26.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:26.443+0000 7f849cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:26.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:26.450+0000 7f849cb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:27.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 12373 12357 1.12245 0.84375 0.02602 0.111254 2024-09-22T03:35:28.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 12505 12489 1.1214 1.03125 0.102925 0.111378 2024-09-22T03:35:28.676 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:28.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:28.835+0000 7f3739c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:28.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:28.841+0000 7f3739c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:29.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 12633 12617 1.12002 1 0.129557 0.111434 2024-09-22T03:35:29.738 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:29.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:35:29.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:29.856+0000 7f50daf17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:29.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:29.866+0000 7f50daf17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:30.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 12764 12748 1.11893 1.02344 0.126236 0.111616 2024-09-22T03:35:30.805 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:35:30.822 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:35:30.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:30.936+0000 7f1c71197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:30.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:30.948+0000 7f1c71197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:30.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:31.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:31.097+0000 7f0f9a22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:31.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:31.102+0000 7f0f9a22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:31.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 15 12904 12889 1.11874 1.10156 0.213109 0.111623 2024-09-22T03:35:31.806 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:35:31.823 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:35:31.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:31.927+0000 7fa94f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:31.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:31.938+0000 7fa94f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:32.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.1664 2024-09-22T03:35:32.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 12904 2024-09-22T03:35:32.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:35:32.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:35:32.207 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.11807 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.100633 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.34375 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 0.84375 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 143 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 12.881 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 172 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 108 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.111697 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.0603991 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.501871 2024-09-22T03:35:32.208 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0135566 2024-09-22T03:35:32.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:35:32.812 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:35:32.839 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:35:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:32.991+0000 7fd1fff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:33.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:33.001+0000 7fd1fff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:33.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:33.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:33.374+0000 7f3c27463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:33.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:33.381+0000 7f3c27463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:33.818 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:35:33.841 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:35:33.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:33.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:33.963+0000 7f712b6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:33.971+0000 7f712b6d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:34.218 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:34.219 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:34.242 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-22T03:35:34.074984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106299955,"num_objects":1617,"num_object_clones":0,"num_object_copies":4851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13043,"num_write_kb":105128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13004,"ondisk_log_size":13004,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":651840,"kb_used_data":651840,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208539072,"statfs":{"total":214211493888,"available":213544009728,"internally_reserved":0,"allocated":667484160,"data_stored":667484160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4554752,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":556,"num_write_kb":4448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000868"},"pg_stats":[{"pgid":"9.7","version":"89'856","reported_seq":874,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826501+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:35:31.826501+0000","last_peered":"2024-09-22T03:35:31.826501+0000","last_clean":"2024-09-22T03:35:31.826501+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:35:31.826501+0000","last_undegraded":"2024-09-22T03:35:31.826501+0000","last_fullsized":"2024-09-22T03:35:31.826501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"89'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829549+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:35:31.829549+0000","last_peered":"2024-09-22T03:35:31.829549+0000","last_clean":"2024-09-22T03:35:31.829549+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:35:31.829549+0000","last_undegraded":"2024-09-22T03:35:31.829549+0000","last_fullsized":"2024-09-22T03:35:31.829549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'728","reported_seq":746,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829046+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:35:31.829046+0000","last_peered":"2024-09-22T03:35:31.829046+0000","last_clean":"2024-09-22T03:35:31.829046+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:35:31.829046+0000","last_undegraded":"2024-09-22T03:35:31.829046+0000","last_fullsized":"2024-09-22T03:35:31.829046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"89'768","reported_seq":786,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829909+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:35:31.829909+0000","last_peered":"2024-09-22T03:35:31.829909+0000","last_clean":"2024-09-22T03:35:31.829909+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:35:31.829909+0000","last_undegraded":"2024-09-22T03:35:31.829909+0000","last_fullsized":"2024-09-22T03:35:31.829909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'896","reported_seq":914,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827738+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:35:31.827738+0000","last_peered":"2024-09-22T03:35:31.827738+0000","last_clean":"2024-09-22T03:35:31.827738+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:35:31.827738+0000","last_undegraded":"2024-09-22T03:35:31.827738+0000","last_fullsized":"2024-09-22T03:35:31.827738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'784","reported_seq":802,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830183+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:35:31.830183+0000","last_peered":"2024-09-22T03:35:31.830183+0000","last_clean":"2024-09-22T03:35:31.830183+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:35:31.830183+0000","last_undegraded":"2024-09-22T03:35:31.830183+0000","last_fullsized":"2024-09-22T03:35:31.830183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'720","reported_seq":738,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829665+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:35:31.829665+0000","last_peered":"2024-09-22T03:35:31.829665+0000","last_clean":"2024-09-22T03:35:31.829665+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:35:31.829665+0000","last_undegraded":"2024-09-22T03:35:31.829665+0000","last_fullsized":"2024-09-22T03:35:31.829665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"89'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.824689+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:35:31.824689+0000","last_peered":"2024-09-22T03:35:31.824689+0000","last_clean":"2024-09-22T03:35:31.824689+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:35:31.824689+0000","last_undegraded":"2024-09-22T03:35:31.824689+0000","last_fullsized":"2024-09-22T03:35:31.824689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'744","reported_seq":762,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.823936+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:35:31.823936+0000","last_peered":"2024-09-22T03:35:31.823936+0000","last_clean":"2024-09-22T03:35:31.823936+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:35:31.823936+0000","last_undegraded":"2024-09-22T03:35:31.823936+0000","last_fullsized":"2024-09-22T03:35:31.823936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":373,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.824429+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:31.824429+0000","last_peered":"2024-09-22T03:35:31.824429+0000","last_clean":"2024-09-22T03:35:31.824429+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:31.824429+0000","last_undegraded":"2024-09-22T03:35:31.824429+0000","last_fullsized":"2024-09-22T03:35:31.824429+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":239,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.828073+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:31.828073+0000","last_peered":"2024-09-22T03:35:31.828073+0000","last_clean":"2024-09-22T03:35:31.828073+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:31.828073+0000","last_undegraded":"2024-09-22T03:35:31.828073+0000","last_fullsized":"2024-09-22T03:35:31.828073+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'848","reported_seq":866,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829912+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:35:31.829912+0000","last_peered":"2024-09-22T03:35:31.829912+0000","last_clean":"2024-09-22T03:35:31.829912+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:35:31.829912+0000","last_undegraded":"2024-09-22T03:35:31.829912+0000","last_fullsized":"2024-09-22T03:35:31.829912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830190+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:31.830190+0000","last_peered":"2024-09-22T03:35:31.830190+0000","last_clean":"2024-09-22T03:35:31.830190+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:31.830190+0000","last_undegraded":"2024-09-22T03:35:31.830190+0000","last_fullsized":"2024-09-22T03:35:31.830190+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'864","reported_seq":882,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.828396+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:35:31.828396+0000","last_peered":"2024-09-22T03:35:31.828396+0000","last_clean":"2024-09-22T03:35:31.828396+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:35:31.828396+0000","last_undegraded":"2024-09-22T03:35:31.828396+0000","last_fullsized":"2024-09-22T03:35:31.828396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826753+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:31.826753+0000","last_peered":"2024-09-22T03:35:31.826753+0000","last_clean":"2024-09-22T03:35:31.826753+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:31.826753+0000","last_undegraded":"2024-09-22T03:35:31.826753+0000","last_fullsized":"2024-09-22T03:35:31.826753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"89'816","reported_seq":834,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827101+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:35:31.827101+0000","last_peered":"2024-09-22T03:35:31.827101+0000","last_clean":"2024-09-22T03:35:31.827101+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:35:31.827101+0000","last_undegraded":"2024-09-22T03:35:31.827101+0000","last_fullsized":"2024-09-22T03:35:31.827101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":240,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826213+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:31.826213+0000","last_peered":"2024-09-22T03:35:31.826213+0000","last_clean":"2024-09-22T03:35:31.826213+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:31.826213+0000","last_undegraded":"2024-09-22T03:35:31.826213+0000","last_fullsized":"2024-09-22T03:35:31.826213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830595+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:35:31.830595+0000","last_peered":"2024-09-22T03:35:31.830595+0000","last_clean":"2024-09-22T03:35:31.830595+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:35:31.830595+0000","last_undegraded":"2024-09-22T03:35:31.830595+0000","last_fullsized":"2024-09-22T03:35:31.830595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829559+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:31.829559+0000","last_peered":"2024-09-22T03:35:31.829559+0000","last_clean":"2024-09-22T03:35:31.829559+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:31.829559+0000","last_undegraded":"2024-09-22T03:35:31.829559+0000","last_fullsized":"2024-09-22T03:35:31.829559+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'856","reported_seq":874,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830678+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:35:31.830678+0000","last_peered":"2024-09-22T03:35:31.830678+0000","last_clean":"2024-09-22T03:35:31.830678+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:35:31.830678+0000","last_undegraded":"2024-09-22T03:35:31.830678+0000","last_fullsized":"2024-09-22T03:35:31.830678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827330+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:31.827330+0000","last_peered":"2024-09-22T03:35:31.827330+0000","last_clean":"2024-09-22T03:35:31.827330+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:31.827330+0000","last_undegraded":"2024-09-22T03:35:31.827330+0000","last_fullsized":"2024-09-22T03:35:31.827330+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'864","reported_seq":882,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827308+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:35:31.827308+0000","last_peered":"2024-09-22T03:35:31.827308+0000","last_clean":"2024-09-22T03:35:31.827308+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:35:31.827308+0000","last_undegraded":"2024-09-22T03:35:31.827308+0000","last_fullsized":"2024-09-22T03:35:31.827308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827646+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:31.827646+0000","last_peered":"2024-09-22T03:35:31.827646+0000","last_clean":"2024-09-22T03:35:31.827646+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:31.827646+0000","last_undegraded":"2024-09-22T03:35:31.827646+0000","last_fullsized":"2024-09-22T03:35:31.827646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"89'664","reported_seq":682,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829309+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:35:31.829309+0000","last_peered":"2024-09-22T03:35:31.829309+0000","last_clean":"2024-09-22T03:35:31.829309+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:35:31.829309+0000","last_undegraded":"2024-09-22T03:35:31.829309+0000","last_fullsized":"2024-09-22T03:35:31.829309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829780+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:31.829780+0000","last_peered":"2024-09-22T03:35:31.829780+0000","last_clean":"2024-09-22T03:35:31.829780+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:31.829780+0000","last_undegraded":"2024-09-22T03:35:31.829780+0000","last_fullsized":"2024-09-22T03:35:31.829780+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":105709568,"num_objects":1613,"num_object_clones":0,"num_object_copies":4839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12904,"num_write_kb":103232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213543854080,"internally_reserved":0,"allocated":667639808,"data_stored":667639808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12904,"ondisk_log_size":12904,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213543854080,"internally_reserved":0,"allocated":667639808,"data_stored":667639808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":175572,"kb_used_data":175572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52122156,"statfs":{"total":53552873472,"available":53373087744,"internally_reserved":0,"allocated":179785728,"data_stored":179785728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":139292,"kb_used_data":139292,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52158436,"statfs":{"total":53552873472,"available":53410238464,"internally_reserved":0,"allocated":142635008,"data_stored":142635008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":133864,"kb_used_data":133864,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163864,"statfs":{"total":53552873472,"available":53415796736,"internally_reserved":0,"allocated":137076736,"data_stored":137076736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":203112,"kb_used_data":203112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52094616,"statfs":{"total":53552873472,"available":53344886784,"internally_reserved":0,"allocated":207986688,"data_stored":207986688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53344862208,"internally_reserved":0,"allocated":208011264,"data_stored":208011264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53415788544,"internally_reserved":0,"allocated":137084928,"data_stored":137084928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53410181120,"internally_reserved":0,"allocated":142692352,"data_stored":142692352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53344862208,"internally_reserved":0,"allocated":208011264,"data_stored":208011264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53415788544,"internally_reserved":0,"allocated":137084928,"data_stored":137084928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53410181120,"internally_reserved":0,"allocated":142692352,"data_stored":142692352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:34.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:34.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:34.382+0000 7f7f7042c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:34.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:34.390+0000 7f7f7042c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:34.621 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:34.621 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:34.644 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-22T03:35:34.074984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106299955,"num_objects":1617,"num_object_clones":0,"num_object_copies":4851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13043,"num_write_kb":105128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13004,"ondisk_log_size":13004,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":651840,"kb_used_data":651840,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208539072,"statfs":{"total":214211493888,"available":213544009728,"internally_reserved":0,"allocated":667484160,"data_stored":667484160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4554752,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":556,"num_write_kb":4448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000868"},"pg_stats":[{"pgid":"9.7","version":"89'856","reported_seq":874,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826501+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:35:31.826501+0000","last_peered":"2024-09-22T03:35:31.826501+0000","last_clean":"2024-09-22T03:35:31.826501+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:35:31.826501+0000","last_undegraded":"2024-09-22T03:35:31.826501+0000","last_fullsized":"2024-09-22T03:35:31.826501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"89'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829549+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:35:31.829549+0000","last_peered":"2024-09-22T03:35:31.829549+0000","last_clean":"2024-09-22T03:35:31.829549+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:35:31.829549+0000","last_undegraded":"2024-09-22T03:35:31.829549+0000","last_fullsized":"2024-09-22T03:35:31.829549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'728","reported_seq":746,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829046+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:35:31.829046+0000","last_peered":"2024-09-22T03:35:31.829046+0000","last_clean":"2024-09-22T03:35:31.829046+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:35:31.829046+0000","last_undegraded":"2024-09-22T03:35:31.829046+0000","last_fullsized":"2024-09-22T03:35:31.829046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"89'768","reported_seq":786,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829909+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:35:31.829909+0000","last_peered":"2024-09-22T03:35:31.829909+0000","last_clean":"2024-09-22T03:35:31.829909+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:35:31.829909+0000","last_undegraded":"2024-09-22T03:35:31.829909+0000","last_fullsized":"2024-09-22T03:35:31.829909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'896","reported_seq":914,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827738+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:35:31.827738+0000","last_peered":"2024-09-22T03:35:31.827738+0000","last_clean":"2024-09-22T03:35:31.827738+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:35:31.827738+0000","last_undegraded":"2024-09-22T03:35:31.827738+0000","last_fullsized":"2024-09-22T03:35:31.827738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'784","reported_seq":802,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830183+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:35:31.830183+0000","last_peered":"2024-09-22T03:35:31.830183+0000","last_clean":"2024-09-22T03:35:31.830183+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:35:31.830183+0000","last_undegraded":"2024-09-22T03:35:31.830183+0000","last_fullsized":"2024-09-22T03:35:31.830183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'720","reported_seq":738,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829665+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:35:31.829665+0000","last_peered":"2024-09-22T03:35:31.829665+0000","last_clean":"2024-09-22T03:35:31.829665+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:35:31.829665+0000","last_undegraded":"2024-09-22T03:35:31.829665+0000","last_fullsized":"2024-09-22T03:35:31.829665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"89'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.824689+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:35:31.824689+0000","last_peered":"2024-09-22T03:35:31.824689+0000","last_clean":"2024-09-22T03:35:31.824689+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:35:31.824689+0000","last_undegraded":"2024-09-22T03:35:31.824689+0000","last_fullsized":"2024-09-22T03:35:31.824689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'744","reported_seq":762,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.823936+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:35:31.823936+0000","last_peered":"2024-09-22T03:35:31.823936+0000","last_clean":"2024-09-22T03:35:31.823936+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:35:31.823936+0000","last_undegraded":"2024-09-22T03:35:31.823936+0000","last_fullsized":"2024-09-22T03:35:31.823936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":373,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.824429+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:31.824429+0000","last_peered":"2024-09-22T03:35:31.824429+0000","last_clean":"2024-09-22T03:35:31.824429+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:31.824429+0000","last_undegraded":"2024-09-22T03:35:31.824429+0000","last_fullsized":"2024-09-22T03:35:31.824429+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":239,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.828073+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:31.828073+0000","last_peered":"2024-09-22T03:35:31.828073+0000","last_clean":"2024-09-22T03:35:31.828073+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:31.828073+0000","last_undegraded":"2024-09-22T03:35:31.828073+0000","last_fullsized":"2024-09-22T03:35:31.828073+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'848","reported_seq":866,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829912+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:35:31.829912+0000","last_peered":"2024-09-22T03:35:31.829912+0000","last_clean":"2024-09-22T03:35:31.829912+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:35:31.829912+0000","last_undegraded":"2024-09-22T03:35:31.829912+0000","last_fullsized":"2024-09-22T03:35:31.829912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830190+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:31.830190+0000","last_peered":"2024-09-22T03:35:31.830190+0000","last_clean":"2024-09-22T03:35:31.830190+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:31.830190+0000","last_undegraded":"2024-09-22T03:35:31.830190+0000","last_fullsized":"2024-09-22T03:35:31.830190+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'864","reported_seq":882,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.828396+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:35:31.828396+0000","last_peered":"2024-09-22T03:35:31.828396+0000","last_clean":"2024-09-22T03:35:31.828396+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:35:31.828396+0000","last_undegraded":"2024-09-22T03:35:31.828396+0000","last_fullsized":"2024-09-22T03:35:31.828396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826753+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:31.826753+0000","last_peered":"2024-09-22T03:35:31.826753+0000","last_clean":"2024-09-22T03:35:31.826753+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:31.826753+0000","last_undegraded":"2024-09-22T03:35:31.826753+0000","last_fullsized":"2024-09-22T03:35:31.826753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"89'816","reported_seq":834,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827101+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:35:31.827101+0000","last_peered":"2024-09-22T03:35:31.827101+0000","last_clean":"2024-09-22T03:35:31.827101+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:35:31.827101+0000","last_undegraded":"2024-09-22T03:35:31.827101+0000","last_fullsized":"2024-09-22T03:35:31.827101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":240,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826213+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:31.826213+0000","last_peered":"2024-09-22T03:35:31.826213+0000","last_clean":"2024-09-22T03:35:31.826213+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:31.826213+0000","last_undegraded":"2024-09-22T03:35:31.826213+0000","last_fullsized":"2024-09-22T03:35:31.826213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830595+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:35:31.830595+0000","last_peered":"2024-09-22T03:35:31.830595+0000","last_clean":"2024-09-22T03:35:31.830595+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:35:31.830595+0000","last_undegraded":"2024-09-22T03:35:31.830595+0000","last_fullsized":"2024-09-22T03:35:31.830595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829559+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:31.829559+0000","last_peered":"2024-09-22T03:35:31.829559+0000","last_clean":"2024-09-22T03:35:31.829559+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:31.829559+0000","last_undegraded":"2024-09-22T03:35:31.829559+0000","last_fullsized":"2024-09-22T03:35:31.829559+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'856","reported_seq":874,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830678+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:35:31.830678+0000","last_peered":"2024-09-22T03:35:31.830678+0000","last_clean":"2024-09-22T03:35:31.830678+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:35:31.830678+0000","last_undegraded":"2024-09-22T03:35:31.830678+0000","last_fullsized":"2024-09-22T03:35:31.830678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827330+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:31.827330+0000","last_peered":"2024-09-22T03:35:31.827330+0000","last_clean":"2024-09-22T03:35:31.827330+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:31.827330+0000","last_undegraded":"2024-09-22T03:35:31.827330+0000","last_fullsized":"2024-09-22T03:35:31.827330+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'864","reported_seq":882,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827308+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:35:31.827308+0000","last_peered":"2024-09-22T03:35:31.827308+0000","last_clean":"2024-09-22T03:35:31.827308+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:35:31.827308+0000","last_undegraded":"2024-09-22T03:35:31.827308+0000","last_fullsized":"2024-09-22T03:35:31.827308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827646+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:31.827646+0000","last_peered":"2024-09-22T03:35:31.827646+0000","last_clean":"2024-09-22T03:35:31.827646+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:31.827646+0000","last_undegraded":"2024-09-22T03:35:31.827646+0000","last_fullsized":"2024-09-22T03:35:31.827646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"89'664","reported_seq":682,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829309+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:35:31.829309+0000","last_peered":"2024-09-22T03:35:31.829309+0000","last_clean":"2024-09-22T03:35:31.829309+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:35:31.829309+0000","last_undegraded":"2024-09-22T03:35:31.829309+0000","last_fullsized":"2024-09-22T03:35:31.829309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829780+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:31.829780+0000","last_peered":"2024-09-22T03:35:31.829780+0000","last_clean":"2024-09-22T03:35:31.829780+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:31.829780+0000","last_undegraded":"2024-09-22T03:35:31.829780+0000","last_fullsized":"2024-09-22T03:35:31.829780+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":105709568,"num_objects":1613,"num_object_clones":0,"num_object_copies":4839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12904,"num_write_kb":103232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213543854080,"internally_reserved":0,"allocated":667639808,"data_stored":667639808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12904,"ondisk_log_size":12904,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213543854080,"internally_reserved":0,"allocated":667639808,"data_stored":667639808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":175572,"kb_used_data":175572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52122156,"statfs":{"total":53552873472,"available":53373087744,"internally_reserved":0,"allocated":179785728,"data_stored":179785728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":139292,"kb_used_data":139292,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52158436,"statfs":{"total":53552873472,"available":53410238464,"internally_reserved":0,"allocated":142635008,"data_stored":142635008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":133864,"kb_used_data":133864,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163864,"statfs":{"total":53552873472,"available":53415796736,"internally_reserved":0,"allocated":137076736,"data_stored":137076736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":203112,"kb_used_data":203112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52094616,"statfs":{"total":53552873472,"available":53344886784,"internally_reserved":0,"allocated":207986688,"data_stored":207986688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53344862208,"internally_reserved":0,"allocated":208011264,"data_stored":208011264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53415788544,"internally_reserved":0,"allocated":137084928,"data_stored":137084928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53410181120,"internally_reserved":0,"allocated":142692352,"data_stored":142692352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53344862208,"internally_reserved":0,"allocated":208011264,"data_stored":208011264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53415788544,"internally_reserved":0,"allocated":137084928,"data_stored":137084928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53410181120,"internally_reserved":0,"allocated":142692352,"data_stored":142692352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:34.646 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:35:34.646 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:34.646 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:34.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:34.808+0000 7fe0fff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:34.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:34.819+0000 7fe0fff97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:35.122 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:35.123 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:35.148 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-22T03:35:34.074984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106299955,"num_objects":1617,"num_object_clones":0,"num_object_copies":4851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":13043,"num_write_kb":105128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13004,"ondisk_log_size":13004,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":651840,"kb_used_data":651840,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208539072,"statfs":{"total":214211493888,"available":213544009728,"internally_reserved":0,"allocated":667484160,"data_stored":667484160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4554752,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":556,"num_write_kb":4448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000868"},"pg_stats":[{"pgid":"9.7","version":"89'856","reported_seq":874,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826501+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:35:31.826501+0000","last_peered":"2024-09-22T03:35:31.826501+0000","last_clean":"2024-09-22T03:35:31.826501+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:35:31.826501+0000","last_undegraded":"2024-09-22T03:35:31.826501+0000","last_fullsized":"2024-09-22T03:35:31.826501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"89'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829549+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:35:31.829549+0000","last_peered":"2024-09-22T03:35:31.829549+0000","last_clean":"2024-09-22T03:35:31.829549+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:35:31.829549+0000","last_undegraded":"2024-09-22T03:35:31.829549+0000","last_fullsized":"2024-09-22T03:35:31.829549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"88'728","reported_seq":746,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829046+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:35:31.829046+0000","last_peered":"2024-09-22T03:35:31.829046+0000","last_clean":"2024-09-22T03:35:31.829046+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:35:31.829046+0000","last_undegraded":"2024-09-22T03:35:31.829046+0000","last_fullsized":"2024-09-22T03:35:31.829046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"89'768","reported_seq":786,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829909+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:35:31.829909+0000","last_peered":"2024-09-22T03:35:31.829909+0000","last_clean":"2024-09-22T03:35:31.829909+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:35:31.829909+0000","last_undegraded":"2024-09-22T03:35:31.829909+0000","last_fullsized":"2024-09-22T03:35:31.829909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"88'896","reported_seq":914,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827738+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:35:31.827738+0000","last_peered":"2024-09-22T03:35:31.827738+0000","last_clean":"2024-09-22T03:35:31.827738+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:35:31.827738+0000","last_undegraded":"2024-09-22T03:35:31.827738+0000","last_fullsized":"2024-09-22T03:35:31.827738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"88'784","reported_seq":802,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830183+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:35:31.830183+0000","last_peered":"2024-09-22T03:35:31.830183+0000","last_clean":"2024-09-22T03:35:31.830183+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:35:31.830183+0000","last_undegraded":"2024-09-22T03:35:31.830183+0000","last_fullsized":"2024-09-22T03:35:31.830183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"88'720","reported_seq":738,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829665+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:35:31.829665+0000","last_peered":"2024-09-22T03:35:31.829665+0000","last_clean":"2024-09-22T03:35:31.829665+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:35:31.829665+0000","last_undegraded":"2024-09-22T03:35:31.829665+0000","last_fullsized":"2024-09-22T03:35:31.829665+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"89'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.824689+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:35:31.824689+0000","last_peered":"2024-09-22T03:35:31.824689+0000","last_clean":"2024-09-22T03:35:31.824689+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:35:31.824689+0000","last_undegraded":"2024-09-22T03:35:31.824689+0000","last_fullsized":"2024-09-22T03:35:31.824689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"88'744","reported_seq":762,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.823936+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:35:31.823936+0000","last_peered":"2024-09-22T03:35:31.823936+0000","last_clean":"2024-09-22T03:35:31.823936+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:35:31.823936+0000","last_undegraded":"2024-09-22T03:35:31.823936+0000","last_fullsized":"2024-09-22T03:35:31.823936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":373,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.824429+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:31.824429+0000","last_peered":"2024-09-22T03:35:31.824429+0000","last_clean":"2024-09-22T03:35:31.824429+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:31.824429+0000","last_undegraded":"2024-09-22T03:35:31.824429+0000","last_fullsized":"2024-09-22T03:35:31.824429+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":239,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.828073+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:31.828073+0000","last_peered":"2024-09-22T03:35:31.828073+0000","last_clean":"2024-09-22T03:35:31.828073+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:31.828073+0000","last_undegraded":"2024-09-22T03:35:31.828073+0000","last_fullsized":"2024-09-22T03:35:31.828073+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"88'848","reported_seq":866,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829912+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:35:31.829912+0000","last_peered":"2024-09-22T03:35:31.829912+0000","last_clean":"2024-09-22T03:35:31.829912+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:35:31.829912+0000","last_undegraded":"2024-09-22T03:35:31.829912+0000","last_fullsized":"2024-09-22T03:35:31.829912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830190+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:31.830190+0000","last_peered":"2024-09-22T03:35:31.830190+0000","last_clean":"2024-09-22T03:35:31.830190+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:31.830190+0000","last_undegraded":"2024-09-22T03:35:31.830190+0000","last_fullsized":"2024-09-22T03:35:31.830190+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"88'864","reported_seq":882,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.828396+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:35:31.828396+0000","last_peered":"2024-09-22T03:35:31.828396+0000","last_clean":"2024-09-22T03:35:31.828396+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:35:31.828396+0000","last_undegraded":"2024-09-22T03:35:31.828396+0000","last_fullsized":"2024-09-22T03:35:31.828396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826753+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:31.826753+0000","last_peered":"2024-09-22T03:35:31.826753+0000","last_clean":"2024-09-22T03:35:31.826753+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:31.826753+0000","last_undegraded":"2024-09-22T03:35:31.826753+0000","last_fullsized":"2024-09-22T03:35:31.826753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"89'816","reported_seq":834,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827101+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:35:31.827101+0000","last_peered":"2024-09-22T03:35:31.827101+0000","last_clean":"2024-09-22T03:35:31.827101+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:35:31.827101+0000","last_undegraded":"2024-09-22T03:35:31.827101+0000","last_fullsized":"2024-09-22T03:35:31.827101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":240,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.826213+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:31.826213+0000","last_peered":"2024-09-22T03:35:31.826213+0000","last_clean":"2024-09-22T03:35:31.826213+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:31.826213+0000","last_undegraded":"2024-09-22T03:35:31.826213+0000","last_fullsized":"2024-09-22T03:35:31.826213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"88'832","reported_seq":850,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830595+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:35:31.830595+0000","last_peered":"2024-09-22T03:35:31.830595+0000","last_clean":"2024-09-22T03:35:31.830595+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:35:31.830595+0000","last_undegraded":"2024-09-22T03:35:31.830595+0000","last_fullsized":"2024-09-22T03:35:31.830595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":832,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829559+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:31.829559+0000","last_peered":"2024-09-22T03:35:31.829559+0000","last_clean":"2024-09-22T03:35:31.829559+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:31.829559+0000","last_undegraded":"2024-09-22T03:35:31.829559+0000","last_fullsized":"2024-09-22T03:35:31.829559+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"88'856","reported_seq":874,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.830678+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:35:31.830678+0000","last_peered":"2024-09-22T03:35:31.830678+0000","last_clean":"2024-09-22T03:35:31.830678+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:35:31.830678+0000","last_undegraded":"2024-09-22T03:35:31.830678+0000","last_fullsized":"2024-09-22T03:35:31.830678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827330+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:31.827330+0000","last_peered":"2024-09-22T03:35:31.827330+0000","last_clean":"2024-09-22T03:35:31.827330+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:31.827330+0000","last_undegraded":"2024-09-22T03:35:31.827330+0000","last_fullsized":"2024-09-22T03:35:31.827330+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"88'864","reported_seq":882,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827308+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:35:31.827308+0000","last_peered":"2024-09-22T03:35:31.827308+0000","last_clean":"2024-09-22T03:35:31.827308+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:35:31.827308+0000","last_undegraded":"2024-09-22T03:35:31.827308+0000","last_fullsized":"2024-09-22T03:35:31.827308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":864,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.827646+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:31.827646+0000","last_peered":"2024-09-22T03:35:31.827646+0000","last_clean":"2024-09-22T03:35:31.827646+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:31.827646+0000","last_undegraded":"2024-09-22T03:35:31.827646+0000","last_fullsized":"2024-09-22T03:35:31.827646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"89'664","reported_seq":682,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829309+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:35:31.829309+0000","last_peered":"2024-09-22T03:35:31.829309+0000","last_clean":"2024-09-22T03:35:31.829309+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:35:31.829309+0000","last_undegraded":"2024-09-22T03:35:31.829309+0000","last_fullsized":"2024-09-22T03:35:31.829309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-22T03:35:31.829780+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:31.829780+0000","last_peered":"2024-09-22T03:35:31.829780+0000","last_clean":"2024-09-22T03:35:31.829780+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:31.829780+0000","last_undegraded":"2024-09-22T03:35:31.829780+0000","last_fullsized":"2024-09-22T03:35:31.829780+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":105709568,"num_objects":1613,"num_object_clones":0,"num_object_copies":4839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12904,"num_write_kb":103232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213543854080,"internally_reserved":0,"allocated":667639808,"data_stored":667639808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12904,"ondisk_log_size":12904,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213543854080,"internally_reserved":0,"allocated":667639808,"data_stored":667639808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":175572,"kb_used_data":175572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52122156,"statfs":{"total":53552873472,"available":53373087744,"internally_reserved":0,"allocated":179785728,"data_stored":179785728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":139292,"kb_used_data":139292,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52158436,"statfs":{"total":53552873472,"available":53410238464,"internally_reserved":0,"allocated":142635008,"data_stored":142635008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":133864,"kb_used_data":133864,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52163864,"statfs":{"total":53552873472,"available":53415796736,"internally_reserved":0,"allocated":137076736,"data_stored":137076736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706413,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":203112,"kb_used_data":203112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52094616,"statfs":{"total":53552873472,"available":53344886784,"internally_reserved":0,"allocated":207986688,"data_stored":207986688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53344862208,"internally_reserved":0,"allocated":208011264,"data_stored":208011264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53415788544,"internally_reserved":0,"allocated":137084928,"data_stored":137084928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53410181120,"internally_reserved":0,"allocated":142692352,"data_stored":142692352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53344862208,"internally_reserved":0,"allocated":208011264,"data_stored":208011264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53415788544,"internally_reserved":0,"allocated":137084928,"data_stored":137084928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53410181120,"internally_reserved":0,"allocated":142692352,"data_stored":142692352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53373022208,"internally_reserved":0,"allocated":179851264,"data_stored":179851264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:35.535 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:35.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:35.666+0000 7f22e7863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:35.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:35.674+0000 7f22e7863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:37.828 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:37.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:37.969+0000 7efda5157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:37.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:37.976+0000 7efda5157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:40.149 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:40.149 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:40.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:40.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:40.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:40.288+0000 7fc2bec63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:40.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:40.304+0000 7fc2bda01640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:40.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:40.344+0000 7f6096418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:40.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:40.349+0000 7f6096418640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:40.627 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:40.628 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:40.651 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-22T03:35:40.075973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":57279051,"num_objects":870,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13792,"num_write_kb":105129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13737,"ondisk_log_size":13737,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":411248,"kb_used_data":411248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208779664,"statfs":{"total":214211493888,"available":213790375936,"internally_reserved":0,"allocated":421117952,"data_stored":421117952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-49020904,"num_objects":-747,"num_object_clones":0,"num_object_copies":-2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":749,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001504"},"pg_stats":[{"pgid":"9.7","version":"92'895","reported_seq":920,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.874784+0000","last_change":"2024-09-22T03:34:00.322017+0000","last_active":"2024-09-22T03:35:36.874784+0000","last_peered":"2024-09-22T03:35:36.874784+0000","last_clean":"2024-09-22T03:35:36.874784+0000","last_became_active":"2024-09-22T03:34:00.277803+0000","last_became_peered":"2024-09-22T03:34:00.277803+0000","last_unstale":"2024-09-22T03:35:36.874784+0000","last_undegraded":"2024-09-22T03:35:36.874784+0000","last_fullsized":"2024-09-22T03:35:36.874784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"92'873","reported_seq":898,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.703598+0000","last_change":"2024-09-22T03:34:00.290394+0000","last_active":"2024-09-22T03:35:36.703598+0000","last_peered":"2024-09-22T03:35:36.703598+0000","last_clean":"2024-09-22T03:35:36.703598+0000","last_became_active":"2024-09-22T03:34:00.248021+0000","last_became_peered":"2024-09-22T03:34:00.248021+0000","last_unstale":"2024-09-22T03:35:36.703598+0000","last_undegraded":"2024-09-22T03:35:36.703598+0000","last_fullsized":"2024-09-22T03:35:36.703598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":874,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"92'774","reported_seq":799,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.904938+0000","last_change":"2024-09-22T03:34:00.379165+0000","last_active":"2024-09-22T03:35:36.904938+0000","last_peered":"2024-09-22T03:35:36.904938+0000","last_clean":"2024-09-22T03:35:36.904938+0000","last_became_active":"2024-09-22T03:34:00.375341+0000","last_became_peered":"2024-09-22T03:34:00.375341+0000","last_unstale":"2024-09-22T03:35:36.904938+0000","last_undegraded":"2024-09-22T03:35:36.904938+0000","last_fullsized":"2024-09-22T03:35:36.904938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"92'812","reported_seq":837,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.809702+0000","last_change":"2024-09-22T03:34:00.248702+0000","last_active":"2024-09-22T03:35:36.809702+0000","last_peered":"2024-09-22T03:35:36.809702+0000","last_clean":"2024-09-22T03:35:36.809702+0000","last_became_active":"2024-09-22T03:34:00.243901+0000","last_became_peered":"2024-09-22T03:34:00.243901+0000","last_unstale":"2024-09-22T03:35:36.809702+0000","last_undegraded":"2024-09-22T03:35:36.809702+0000","last_fullsized":"2024-09-22T03:35:36.809702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":813,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"92'942","reported_seq":967,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.937290+0000","last_change":"2024-09-22T03:34:00.343478+0000","last_active":"2024-09-22T03:35:36.937290+0000","last_peered":"2024-09-22T03:35:36.937290+0000","last_clean":"2024-09-22T03:35:36.937290+0000","last_became_active":"2024-09-22T03:34:00.339449+0000","last_became_peered":"2024-09-22T03:34:00.339449+0000","last_unstale":"2024-09-22T03:35:36.937290+0000","last_undegraded":"2024-09-22T03:35:36.937290+0000","last_fullsized":"2024-09-22T03:35:36.937290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":942,"log_dups_size":0,"ondisk_log_size":942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":943,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2","version":"92'823","reported_seq":848,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.777712+0000","last_change":"2024-09-22T03:34:00.305417+0000","last_active":"2024-09-22T03:35:36.777712+0000","last_peered":"2024-09-22T03:35:36.777712+0000","last_clean":"2024-09-22T03:35:36.777712+0000","last_became_active":"2024-09-22T03:34:00.299314+0000","last_became_peered":"2024-09-22T03:34:00.299314+0000","last_unstale":"2024-09-22T03:35:36.777712+0000","last_undegraded":"2024-09-22T03:35:36.777712+0000","last_fullsized":"2024-09-22T03:35:36.777712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":824,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"92'766","reported_seq":791,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.788332+0000","last_change":"2024-09-22T03:34:00.262850+0000","last_active":"2024-09-22T03:35:36.788332+0000","last_peered":"2024-09-22T03:35:36.788332+0000","last_clean":"2024-09-22T03:35:36.788332+0000","last_became_active":"2024-09-22T03:34:00.250884+0000","last_became_peered":"2024-09-22T03:34:00.250884+0000","last_unstale":"2024-09-22T03:35:36.788332+0000","last_undegraded":"2024-09-22T03:35:36.788332+0000","last_fullsized":"2024-09-22T03:35:36.788332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":767,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"92'883","reported_seq":908,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.865614+0000","last_change":"2024-09-22T03:34:00.264015+0000","last_active":"2024-09-22T03:35:36.865614+0000","last_peered":"2024-09-22T03:35:36.865614+0000","last_clean":"2024-09-22T03:35:36.865614+0000","last_became_active":"2024-09-22T03:34:00.231561+0000","last_became_peered":"2024-09-22T03:34:00.231561+0000","last_unstale":"2024-09-22T03:35:36.865614+0000","last_undegraded":"2024-09-22T03:35:36.865614+0000","last_fullsized":"2024-09-22T03:35:36.865614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":884,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"92'779","reported_seq":804,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.763676+0000","last_change":"2024-09-22T03:34:00.268020+0000","last_active":"2024-09-22T03:35:36.763676+0000","last_peered":"2024-09-22T03:35:36.763676+0000","last_clean":"2024-09-22T03:35:36.763676+0000","last_became_active":"2024-09-22T03:34:00.247582+0000","last_became_peered":"2024-09-22T03:34:00.247582+0000","last_unstale":"2024-09-22T03:35:36.763676+0000","last_undegraded":"2024-09-22T03:35:36.763676+0000","last_fullsized":"2024-09-22T03:35:36.763676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":780,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":379,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:33.869826+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:33.869826+0000","last_peered":"2024-09-22T03:35:33.869826+0000","last_clean":"2024-09-22T03:35:33.869826+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:33.869826+0000","last_undegraded":"2024-09-22T03:35:33.869826+0000","last_fullsized":"2024-09-22T03:35:33.869826+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":245,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:33.872426+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:33.872426+0000","last_peered":"2024-09-22T03:35:33.872426+0000","last_clean":"2024-09-22T03:35:33.872426+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:33.872426+0000","last_undegraded":"2024-09-22T03:35:33.872426+0000","last_fullsized":"2024-09-22T03:35:33.872426+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"92'900","reported_seq":925,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.720441+0000","last_change":"2024-09-22T03:34:00.266499+0000","last_active":"2024-09-22T03:35:36.720441+0000","last_peered":"2024-09-22T03:35:36.720441+0000","last_clean":"2024-09-22T03:35:36.720441+0000","last_became_active":"2024-09-22T03:34:00.256214+0000","last_became_peered":"2024-09-22T03:34:00.256214+0000","last_unstale":"2024-09-22T03:35:36.720441+0000","last_undegraded":"2024-09-22T03:35:36.720441+0000","last_fullsized":"2024-09-22T03:35:36.720441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":901,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:34.050061+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:34.050061+0000","last_peered":"2024-09-22T03:35:34.050061+0000","last_clean":"2024-09-22T03:35:34.050061+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:34.050061+0000","last_undegraded":"2024-09-22T03:35:34.050061+0000","last_fullsized":"2024-09-22T03:35:34.050061+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"92'923","reported_seq":948,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.832906+0000","last_change":"2024-09-22T03:34:00.309400+0000","last_active":"2024-09-22T03:35:36.832906+0000","last_peered":"2024-09-22T03:35:36.832906+0000","last_clean":"2024-09-22T03:35:36.832906+0000","last_became_active":"2024-09-22T03:34:00.304561+0000","last_became_peered":"2024-09-22T03:34:00.304561+0000","last_unstale":"2024-09-22T03:35:36.832906+0000","last_undegraded":"2024-09-22T03:35:36.832906+0000","last_fullsized":"2024-09-22T03:35:36.832906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":923,"log_dups_size":0,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":924,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:33.867977+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:33.867977+0000","last_peered":"2024-09-22T03:35:33.867977+0000","last_clean":"2024-09-22T03:35:33.867977+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:33.867977+0000","last_undegraded":"2024-09-22T03:35:33.867977+0000","last_fullsized":"2024-09-22T03:35:33.867977+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"92'859","reported_seq":884,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.854546+0000","last_change":"2024-09-22T03:34:00.325619+0000","last_active":"2024-09-22T03:35:36.854546+0000","last_peered":"2024-09-22T03:35:36.854546+0000","last_clean":"2024-09-22T03:35:36.854546+0000","last_became_active":"2024-09-22T03:34:00.318784+0000","last_became_peered":"2024-09-22T03:34:00.318784+0000","last_unstale":"2024-09-22T03:35:36.854546+0000","last_undegraded":"2024-09-22T03:35:36.854546+0000","last_fullsized":"2024-09-22T03:35:36.854546+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":860,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:34.047913+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:34.047913+0000","last_peered":"2024-09-22T03:35:34.047913+0000","last_clean":"2024-09-22T03:35:34.047913+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:34.047913+0000","last_undegraded":"2024-09-22T03:35:34.047913+0000","last_fullsized":"2024-09-22T03:35:34.047913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"92'884","reported_seq":909,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.897182+0000","last_change":"2024-09-22T03:34:00.314534+0000","last_active":"2024-09-22T03:35:36.897182+0000","last_peered":"2024-09-22T03:35:36.897182+0000","last_clean":"2024-09-22T03:35:36.897182+0000","last_became_active":"2024-09-22T03:34:00.260107+0000","last_became_peered":"2024-09-22T03:34:00.260107+0000","last_unstale":"2024-09-22T03:35:36.897182+0000","last_undegraded":"2024-09-22T03:35:36.897182+0000","last_fullsized":"2024-09-22T03:35:36.897182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":885,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:33.848820+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:33.848820+0000","last_peered":"2024-09-22T03:35:33.848820+0000","last_clean":"2024-09-22T03:35:33.848820+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:33.848820+0000","last_undegraded":"2024-09-22T03:35:33.848820+0000","last_fullsized":"2024-09-22T03:35:33.848820+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.e","version":"92'900","reported_seq":925,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.819172+0000","last_change":"2024-09-22T03:34:00.331565+0000","last_active":"2024-09-22T03:35:36.819172+0000","last_peered":"2024-09-22T03:35:36.819172+0000","last_clean":"2024-09-22T03:35:36.819172+0000","last_became_active":"2024-09-22T03:34:00.326686+0000","last_became_peered":"2024-09-22T03:34:00.326686+0000","last_unstale":"2024-09-22T03:35:36.819172+0000","last_undegraded":"2024-09-22T03:35:36.819172+0000","last_fullsized":"2024-09-22T03:35:36.819172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":901,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:33.873197+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:33.873197+0000","last_peered":"2024-09-22T03:35:33.873197+0000","last_clean":"2024-09-22T03:35:33.873197+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:33.873197+0000","last_undegraded":"2024-09-22T03:35:33.873197+0000","last_fullsized":"2024-09-22T03:35:33.873197+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"92'920","reported_seq":946,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.862375+0000","last_change":"2024-09-22T03:34:00.260601+0000","last_active":"2024-09-22T03:35:36.862375+0000","last_peered":"2024-09-22T03:35:36.862375+0000","last_clean":"2024-09-22T03:35:36.862375+0000","last_became_active":"2024-09-22T03:34:00.254533+0000","last_became_peered":"2024-09-22T03:34:00.254533+0000","last_unstale":"2024-09-22T03:35:36.862375+0000","last_undegraded":"2024-09-22T03:35:36.862375+0000","last_fullsized":"2024-09-22T03:35:36.862375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407896,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":921,"num_write_kb":6913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:33.846253+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:33.846253+0000","last_peered":"2024-09-22T03:35:33.846253+0000","last_clean":"2024-09-22T03:35:33.846253+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:33.846253+0000","last_undegraded":"2024-09-22T03:35:33.846253+0000","last_fullsized":"2024-09-22T03:35:33.846253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.c","version":"92'704","reported_seq":729,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:36.894860+0000","last_change":"2024-09-22T03:34:00.348671+0000","last_active":"2024-09-22T03:35:36.894860+0000","last_peered":"2024-09-22T03:35:36.894860+0000","last_clean":"2024-09-22T03:35:36.894860+0000","last_became_active":"2024-09-22T03:34:00.342592+0000","last_became_peered":"2024-09-22T03:34:00.342592+0000","last_unstale":"2024-09-22T03:35:36.894860+0000","last_undegraded":"2024-09-22T03:35:36.894860+0000","last_fullsized":"2024-09-22T03:35:36.894860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:33:59.113113+0000","last_clean_scrub_stamp":"2024-09-22T03:33:59.113113+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":705,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-22T03:35:33.872257+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:33.872257+0000","last_peered":"2024-09-22T03:35:33.872257+0000","last_clean":"2024-09-22T03:35:33.872257+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:33.872257+0000","last_undegraded":"2024-09-22T03:35:33.872257+0000","last_fullsized":"2024-09-22T03:35:33.872257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":56688664,"num_objects":866,"num_object_clones":0,"num_object_copies":2598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13653,"num_write_kb":103233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213835755520,"internally_reserved":0,"allocated":375738368,"data_stored":375738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13637,"ondisk_log_size":13637,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213835755520,"internally_reserved":0,"allocated":375738368,"data_stored":375738368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53451915264,"internally_reserved":0,"allocated":100958208,"data_stored":100958208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":111080,"kb_used_data":111080,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52186648,"statfs":{"total":53552873472,"available":53439127552,"internally_reserved":0,"allocated":113745920,"data_stored":113745920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":87656,"kb_used_data":87656,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52210072,"statfs":{"total":53552873472,"available":53463113728,"internally_reserved":0,"allocated":89759744,"data_stored":89759744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":85588,"kb_used_data":85588,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52212140,"statfs":{"total":53552873472,"available":53465231360,"internally_reserved":0,"allocated":87642112,"data_stored":87642112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":126924,"kb_used_data":126924,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52170804,"statfs":{"total":53552873472,"available":53422903296,"internally_reserved":0,"allocated":129970176,"data_stored":129970176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53451915264,"internally_reserved":0,"allocated":100958208,"data_stored":100958208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53436325888,"internally_reserved":0,"allocated":116547584,"data_stored":116547584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53474476032,"internally_reserved":0,"allocated":78397440,"data_stored":78397440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53472907264,"internally_reserved":0,"allocated":79966208,"data_stored":79966208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53452046336,"internally_reserved":0,"allocated":100827136,"data_stored":100827136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":53552873472,"available":53436325888,"internally_reserved":0,"allocated":116547584,"data_stored":116547584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":53552873472,"available":53474476032,"internally_reserved":0,"allocated":78397440,"data_stored":78397440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":53552873472,"available":53472907264,"internally_reserved":0,"allocated":79966208,"data_stored":79966208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":53552873472,"available":53452046336,"internally_reserved":0,"allocated":100827136,"data_stored":100827136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:42.533 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:42.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:42.647+0000 7fe92a557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:42.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:42.652+0000 7fe92a557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:43.224 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 1613 objects 2024-09-22T03:35:43.225 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :10.9953 2024-09-22T03:35:44.820 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:44.869 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_6 2024-09-22T03:35:44.869 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_6 unique_pool_6 --yes-i-really-really-mean-it 2024-09-22T03:35:44.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:44.935+0000 7f177ee63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:44.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:44.940+0000 7f177ee63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:44.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:44.985+0000 7f70ac357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:45.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:45.001+0000 7f70ac357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:45.654 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:45.654 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:45.654 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:35:45.654 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:35:45.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:45.754+0000 7fee9fe63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:45.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:45.761+0000 7fee9fe63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:45.775 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:45.775 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:45.775 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:45.776 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:45.778 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:45.779 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:45.779 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:35:45.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:45.877+0000 7fcc39b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:45.885+0000 7fcc39b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:45.899 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:45.899 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:45.899 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:45.900 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:45.903 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:45.903 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:45.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:35:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:46.001+0000 7f3041997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:46.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:46.009+0000 7f3041997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:46.025 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:46.025 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:46.025 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:46.025 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:46.025 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:46.025 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:46.025 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:46.026 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:46.026 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:46.026 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:46.026 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:46.026 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:46.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:46.028 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:46.029 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:35:46.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:46.127+0000 7efcc426c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:46.133 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_6' removed 2024-09-22T03:35:46.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:46.136+0000 7efcc426c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:46.156 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-22T03:35:46.156 INFO:tasks.radosbench:Beginning radosbench... 2024-09-22T03:35:46.156 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_7 2024-09-22T03:35:46.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_7 16 2024-09-22T03:35:46.160 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:35:46.160 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:35:46.161 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:35:46.166 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:35:46.166 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:35:46.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:46.284+0000 7f0291d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:46.296+0000 7f0291d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:47.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:47.134 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_7' created 2024-09-22T03:35:47.152 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_7 rados --yes-i-really-mean-it || true 2024-09-22T03:35:47.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:47.156+0000 7fe46dd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:47.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:47.163+0000 7fe46dd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:47.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:47.281+0000 7f4ec5197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:47.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:47.292+0000 7f4ec5197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:48.140 INFO:teuthology.orchestra.run.smithi129.stderr:enabled application 'rados' on pool 'unique_pool_7' 2024-09-22T03:35:49.159 DEBUG:teuthology.orchestra.run.smithi129:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_7 bench 90 write' 2024-09-22T03:35:49.161 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-22T03:35:49.162 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-22T03:35:49.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:hints = 1 2024-09-22T03:35:49.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-22T03:35:49.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object prefix: benchmark_data_smithi129_62826 2024-09-22T03:35:49.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:35:49.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 0 0 0 0 0 0 - 0 2024-09-22T03:35:49.282 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:49.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:49.415+0000 7fcbf3f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:49.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:49.421+0000 7fcbf3f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:50.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 1 16 150 134 1.04679 1.04688 0.21639 0.105664 2024-09-22T03:35:51.167 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:51.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:51.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:51.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 2 16 278 262 1.02335 1 0.17921 0.114713 2024-09-22T03:35:51.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:51.278+0000 7f6114d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:51.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:51.286+0000 7f6114d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:51.525 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:51.526 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:51.546 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-22T03:35:50.077868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":24376,"kb_used_data":24376,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209166536,"statfs":{"total":214211493888,"available":214186532864,"internally_reserved":0,"allocated":24961024,"data_stored":24961024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9502744,"num_objects":-146,"num_object_clones":0,"num_object_copies":-438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-14373,"num_write_kb":-103233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000721"},"pg_stats":[{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":382,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.170226+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:46.170226+0000","last_peered":"2024-09-22T03:35:46.170226+0000","last_clean":"2024-09-22T03:35:46.170226+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:46.170226+0000","last_undegraded":"2024-09-22T03:35:46.170226+0000","last_fullsized":"2024-09-22T03:35:46.170226+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":248,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.166466+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:46.166466+0000","last_peered":"2024-09-22T03:35:46.166466+0000","last_clean":"2024-09-22T03:35:46.166466+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:46.166466+0000","last_undegraded":"2024-09-22T03:35:46.166466+0000","last_fullsized":"2024-09-22T03:35:46.166466+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":247,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.177050+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:46.177050+0000","last_peered":"2024-09-22T03:35:46.177050+0000","last_clean":"2024-09-22T03:35:46.177050+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:46.177050+0000","last_undegraded":"2024-09-22T03:35:46.177050+0000","last_fullsized":"2024-09-22T03:35:46.177050+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.160025+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:46.160025+0000","last_peered":"2024-09-22T03:35:46.160025+0000","last_clean":"2024-09-22T03:35:46.160025+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:46.160025+0000","last_undegraded":"2024-09-22T03:35:46.160025+0000","last_fullsized":"2024-09-22T03:35:46.160025+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":249,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.171081+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:46.171081+0000","last_peered":"2024-09-22T03:35:46.171081+0000","last_clean":"2024-09-22T03:35:46.171081+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:46.171081+0000","last_undegraded":"2024-09-22T03:35:46.171081+0000","last_fullsized":"2024-09-22T03:35:46.171081+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":247,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.178574+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:46.178574+0000","last_peered":"2024-09-22T03:35:46.178574+0000","last_clean":"2024-09-22T03:35:46.178574+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:46.178574+0000","last_undegraded":"2024-09-22T03:35:46.178574+0000","last_fullsized":"2024-09-22T03:35:46.178574+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":247,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.170710+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:46.170710+0000","last_peered":"2024-09-22T03:35:46.170710+0000","last_clean":"2024-09-22T03:35:46.170710+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:46.170710+0000","last_undegraded":"2024-09-22T03:35:46.170710+0000","last_fullsized":"2024-09-22T03:35:46.170710+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":247,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.169938+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:46.169938+0000","last_peered":"2024-09-22T03:35:46.169938+0000","last_clean":"2024-09-22T03:35:46.169938+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:46.169938+0000","last_undegraded":"2024-09-22T03:35:46.169938+0000","last_fullsized":"2024-09-22T03:35:46.169938+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-22T03:35:47.133123+0000","last_change":"2024-09-22T03:35:47.133123+0000","last_active":"2024-09-22T03:35:47.133123+0000","last_peered":"2024-09-22T03:35:47.133123+0000","last_clean":"2024-09-22T03:35:47.133123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-22T03:35:47.133123+0000","last_undegraded":"2024-09-22T03:35:47.133123+0000","last_fullsized":"2024-09-22T03:35:47.133123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":247,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-22T03:35:46.167815+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:46.167815+0000","last_peered":"2024-09-22T03:35:46.167815+0000","last_clean":"2024-09-22T03:35:46.167815+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:46.167815+0000","last_undegraded":"2024-09-22T03:35:46.167815+0000","last_fullsized":"2024-09-22T03:35:46.167815+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214202900480,"internally_reserved":0,"allocated":8593408,"data_stored":8593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550632960,"internally_reserved":0,"allocated":2240512,"data_stored":2240512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8128,"kb_used_data":8128,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289600,"statfs":{"total":53552873472,"available":53544550400,"internally_reserved":0,"allocated":8323072,"data_stored":8323072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4240,"kb_used_data":4240,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293488,"statfs":{"total":53552873472,"available":53548531712,"internally_reserved":0,"allocated":4341760,"data_stored":4341760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3772,"kb_used_data":3772,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293956,"statfs":{"total":53552873472,"available":53549010944,"internally_reserved":0,"allocated":3862528,"data_stored":3862528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8236,"kb_used_data":8236,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289492,"statfs":{"total":53552873472,"available":53544439808,"internally_reserved":0,"allocated":8433664,"data_stored":8433664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550632960,"internally_reserved":0,"allocated":2240512,"data_stored":2240512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550522368,"internally_reserved":0,"allocated":2351104,"data_stored":2351104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551345664,"internally_reserved":0,"allocated":1527808,"data_stored":1527808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550399488,"internally_reserved":0,"allocated":2473984,"data_stored":2473984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550632960,"internally_reserved":0,"allocated":2240512,"data_stored":2240512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:51.572 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:51.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:51.682+0000 7f1d3b717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:51.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:51.688+0000 7f1d3b717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:52.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 3 16 398 382 0.994705 0.9375 0.219771 0.121367 2024-09-22T03:35:53.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 4 16 545 529 1.03312 1.14844 0.210303 0.117683 2024-09-22T03:35:53.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:53.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:53.903+0000 7f8ae086c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:53.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:53.909+0000 7f8ae086c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:54.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 5 16 692 676 1.05616 1.14844 0.0940942 0.116755 2024-09-22T03:35:55.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 6 16 842 826 1.07543 1.17188 0.0535075 0.115125 2024-09-22T03:35:56.137 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:35:56.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 7 16 977 961 1.07246 1.05469 0.0852218 0.114615 2024-09-22T03:35:56.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:56.253+0000 7f5b27a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:56.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:56.260+0000 7f5b27a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:56.547 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:35:56.548 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:35:56.548 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:35:56.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:56.658+0000 7f0a98197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:56.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:56.665+0000 7f0a98197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:35:56.904 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:35:56.919 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-22T03:35:56.078955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3605043,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":507,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":454,"ondisk_log_size":454,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":24956,"kb_used_data":24956,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209165956,"statfs":{"total":214211493888,"available":214185938944,"internally_reserved":0,"allocated":25554944,"data_stored":25554944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001449"},"pg_stats":[{"pgid":"10.7","version":"96'15","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:50.514271+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:35:50.514271+0000","last_peered":"2024-09-22T03:35:50.514271+0000","last_clean":"2024-09-22T03:35:50.514271+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:35:50.514271+0000","last_undegraded":"2024-09-22T03:35:50.514271+0000","last_fullsized":"2024-09-22T03:35:50.514271+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"96'47","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.980169+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:35:51.980169+0000","last_peered":"2024-09-22T03:35:51.980169+0000","last_clean":"2024-09-22T03:35:51.980169+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:35:51.980169+0000","last_undegraded":"2024-09-22T03:35:51.980169+0000","last_fullsized":"2024-09-22T03:35:51.980169+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"96'31","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.131078+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:35:51.131078+0000","last_peered":"2024-09-22T03:35:51.131078+0000","last_clean":"2024-09-22T03:35:51.131078+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:35:51.131078+0000","last_undegraded":"2024-09-22T03:35:51.131078+0000","last_fullsized":"2024-09-22T03:35:51.131078+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"96'23","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.487157+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:35:49.487157+0000","last_peered":"2024-09-22T03:35:49.487157+0000","last_clean":"2024-09-22T03:35:49.487157+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:35:49.487157+0000","last_undegraded":"2024-09-22T03:35:49.487157+0000","last_fullsized":"2024-09-22T03:35:49.487157+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"96'39","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.928361+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:35:51.928361+0000","last_peered":"2024-09-22T03:35:51.928361+0000","last_clean":"2024-09-22T03:35:51.928361+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:35:51.928361+0000","last_undegraded":"2024-09-22T03:35:51.928361+0000","last_fullsized":"2024-09-22T03:35:51.928361+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.169201+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:35:49.169201+0000","last_peered":"2024-09-22T03:35:49.169201+0000","last_clean":"2024-09-22T03:35:49.169201+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:35:49.169201+0000","last_undegraded":"2024-09-22T03:35:49.169201+0000","last_fullsized":"2024-09-22T03:35:49.169201+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"96'31","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.537164+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:35:51.537164+0000","last_peered":"2024-09-22T03:35:51.537164+0000","last_clean":"2024-09-22T03:35:51.537164+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:35:51.537164+0000","last_undegraded":"2024-09-22T03:35:51.537164+0000","last_fullsized":"2024-09-22T03:35:51.537164+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"96'31","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.418739+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:35:51.418739+0000","last_peered":"2024-09-22T03:35:51.418739+0000","last_clean":"2024-09-22T03:35:51.418739+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:35:51.418739+0000","last_undegraded":"2024-09-22T03:35:51.418739+0000","last_fullsized":"2024-09-22T03:35:51.418739+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"96'31","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:50.931589+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:35:50.931589+0000","last_peered":"2024-09-22T03:35:50.931589+0000","last_clean":"2024-09-22T03:35:50.931589+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:35:50.931589+0000","last_undegraded":"2024-09-22T03:35:50.931589+0000","last_fullsized":"2024-09-22T03:35:50.931589+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":391,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.163695+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:49.163695+0000","last_peered":"2024-09-22T03:35:49.163695+0000","last_clean":"2024-09-22T03:35:49.163695+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:49.163695+0000","last_undegraded":"2024-09-22T03:35:49.163695+0000","last_fullsized":"2024-09-22T03:35:49.163695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":257,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.170082+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:49.170082+0000","last_peered":"2024-09-22T03:35:49.170082+0000","last_clean":"2024-09-22T03:35:49.170082+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:49.170082+0000","last_undegraded":"2024-09-22T03:35:49.170082+0000","last_fullsized":"2024-09-22T03:35:49.170082+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"96'23","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.842175+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:35:49.842175+0000","last_peered":"2024-09-22T03:35:49.842175+0000","last_clean":"2024-09-22T03:35:49.842175+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:35:49.842175+0000","last_undegraded":"2024-09-22T03:35:49.842175+0000","last_fullsized":"2024-09-22T03:35:49.842175+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.169575+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:49.169575+0000","last_peered":"2024-09-22T03:35:49.169575+0000","last_clean":"2024-09-22T03:35:49.169575+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:49.169575+0000","last_undegraded":"2024-09-22T03:35:49.169575+0000","last_fullsized":"2024-09-22T03:35:49.169575+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"96'15","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.313959+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:35:51.313959+0000","last_peered":"2024-09-22T03:35:51.313959+0000","last_clean":"2024-09-22T03:35:51.313959+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:35:51.313959+0000","last_undegraded":"2024-09-22T03:35:51.313959+0000","last_fullsized":"2024-09-22T03:35:51.313959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.165538+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:49.165538+0000","last_peered":"2024-09-22T03:35:49.165538+0000","last_clean":"2024-09-22T03:35:49.165538+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:49.165538+0000","last_undegraded":"2024-09-22T03:35:49.165538+0000","last_fullsized":"2024-09-22T03:35:49.165538+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"96'15","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.355156+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:35:51.355156+0000","last_peered":"2024-09-22T03:35:51.355156+0000","last_clean":"2024-09-22T03:35:51.355156+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:35:51.355156+0000","last_undegraded":"2024-09-22T03:35:51.355156+0000","last_fullsized":"2024-09-22T03:35:51.355156+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":258,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.165927+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:49.165927+0000","last_peered":"2024-09-22T03:35:49.165927+0000","last_clean":"2024-09-22T03:35:49.165927+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:49.165927+0000","last_undegraded":"2024-09-22T03:35:49.165927+0000","last_fullsized":"2024-09-22T03:35:49.165927+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"96'39","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.900747+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:35:51.900747+0000","last_peered":"2024-09-22T03:35:51.900747+0000","last_clean":"2024-09-22T03:35:51.900747+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:35:51.900747+0000","last_undegraded":"2024-09-22T03:35:51.900747+0000","last_fullsized":"2024-09-22T03:35:51.900747+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.168040+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:49.168040+0000","last_peered":"2024-09-22T03:35:49.168040+0000","last_clean":"2024-09-22T03:35:49.168040+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:49.168040+0000","last_undegraded":"2024-09-22T03:35:49.168040+0000","last_fullsized":"2024-09-22T03:35:49.168040+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.173102+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:35:49.173102+0000","last_peered":"2024-09-22T03:35:49.173102+0000","last_clean":"2024-09-22T03:35:49.173102+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:35:49.173102+0000","last_undegraded":"2024-09-22T03:35:49.173102+0000","last_fullsized":"2024-09-22T03:35:49.173102+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.169317+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:49.169317+0000","last_peered":"2024-09-22T03:35:49.169317+0000","last_clean":"2024-09-22T03:35:49.169317+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:49.169317+0000","last_undegraded":"2024-09-22T03:35:49.169317+0000","last_fullsized":"2024-09-22T03:35:49.169317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"96'7","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:51.739377+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:35:51.739377+0000","last_peered":"2024-09-22T03:35:51.739377+0000","last_clean":"2024-09-22T03:35:51.739377+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:35:51.739377+0000","last_undegraded":"2024-09-22T03:35:51.739377+0000","last_fullsized":"2024-09-22T03:35:51.739377+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.167860+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:49.167860+0000","last_peered":"2024-09-22T03:35:49.167860+0000","last_clean":"2024-09-22T03:35:49.167860+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:49.167860+0000","last_undegraded":"2024-09-22T03:35:49.167860+0000","last_fullsized":"2024-09-22T03:35:49.167860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"96'7","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.919639+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:35:49.919639+0000","last_peered":"2024-09-22T03:35:49.919639+0000","last_clean":"2024-09-22T03:35:49.919639+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:35:49.919639+0000","last_undegraded":"2024-09-22T03:35:49.919639+0000","last_fullsized":"2024-09-22T03:35:49.919639+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.172060+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:49.172060+0000","last_peered":"2024-09-22T03:35:49.172060+0000","last_clean":"2024-09-22T03:35:49.172060+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:49.172060+0000","last_undegraded":"2024-09-22T03:35:49.172060+0000","last_fullsized":"2024-09-22T03:35:49.172060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214181556224,"internally_reserved":0,"allocated":29937664,"data_stored":29937664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":354,"ondisk_log_size":354,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214181629952,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53546348544,"internally_reserved":0,"allocated":6524928,"data_stored":6524928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5540,"kb_used_data":5540,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292188,"statfs":{"total":53552873472,"available":53547200512,"internally_reserved":0,"allocated":5672960,"data_stored":5672960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7564,"kb_used_data":7564,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290164,"statfs":{"total":53552873472,"available":53545127936,"internally_reserved":0,"allocated":7745536,"data_stored":7745536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5496,"kb_used_data":5496,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292232,"statfs":{"total":53552873472,"available":53547245568,"internally_reserved":0,"allocated":5627904,"data_stored":5627904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706433,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6356,"kb_used_data":6356,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52291372,"statfs":{"total":53552873472,"available":53546364928,"internally_reserved":0,"allocated":6508544,"data_stored":6508544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53546348544,"internally_reserved":0,"allocated":6524928,"data_stored":6524928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53545082880,"internally_reserved":0,"allocated":7790592,"data_stored":7790592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53546500096,"internally_reserved":0,"allocated":6373376,"data_stored":6373376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53543698432,"internally_reserved":0,"allocated":9175040,"data_stored":9175040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53546348544,"internally_reserved":0,"allocated":6524928,"data_stored":6524928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53545082880,"internally_reserved":0,"allocated":7790592,"data_stored":7790592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53546500096,"internally_reserved":0,"allocated":6373376,"data_stored":6373376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53543624704,"internally_reserved":0,"allocated":9248768,"data_stored":9248768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53546348544,"internally_reserved":0,"allocated":6524928,"data_stored":6524928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:35:57.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 8 16 1103 1087 1.06144 0.984375 0.104664 0.116825 2024-09-22T03:35:58.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 9 16 1244 1228 1.06588 1.10156 0.128596 0.11624 2024-09-22T03:35:58.410 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:35:58.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:58.552+0000 7f03160a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:58.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:35:58.557+0000 7f03160a9640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:35:59.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 10 16 1376 1360 1.06241 1.03125 0.0740848 0.116931 2024-09-22T03:36:00.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 11 16 1504 1488 1.05673 1 0.190398 0.11724 2024-09-22T03:36:00.730 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:00.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:00.839+0000 7f4cbc4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:00.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:00.844+0000 7f4cbc4d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:01.227 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 12 16 1651 1635 1.06437 1.14844 0.125378 0.116877 2024-09-22T03:36:01.922 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:01.922 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:01.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:36:02.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:02.037+0000 7fa854197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:02.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:02.045+0000 7fa854197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:02.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 13 16 1785 1769 1.06301 1.04688 0.109307 0.116902 2024-09-22T03:36:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:36:02.362 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:36:02.376 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-22T03:36:02.080034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":14967347,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1894,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1839,"ondisk_log_size":1839,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":91548,"kb_used_data":91548,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209099364,"statfs":{"total":214211493888,"available":214117748736,"internally_reserved":0,"allocated":93745152,"data_stored":93745152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14376960,"num_objects":220,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":14040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002165"},"pg_stats":[{"pgid":"10.7","version":"96'98","reported_seq":111,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.992496+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:36:01.992496+0000","last_peered":"2024-09-22T03:36:01.992496+0000","last_clean":"2024-09-22T03:36:01.992496+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:36:01.992496+0000","last_undegraded":"2024-09-22T03:36:01.992496+0000","last_fullsized":"2024-09-22T03:36:01.992496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":811008,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"96'135","reported_seq":148,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.485839+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:36:01.485839+0000","last_peered":"2024-09-22T03:36:01.485839+0000","last_clean":"2024-09-22T03:36:01.485839+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:36:01.485839+0000","last_undegraded":"2024-09-22T03:36:01.485839+0000","last_fullsized":"2024-09-22T03:36:01.485839+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"96'143","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.458318+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:36:01.458318+0000","last_peered":"2024-09-22T03:36:01.458318+0000","last_clean":"2024-09-22T03:36:01.458318+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:36:01.458318+0000","last_undegraded":"2024-09-22T03:36:01.458318+0000","last_fullsized":"2024-09-22T03:36:01.458318+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"96'143","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.714468+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:36:01.714468+0000","last_peered":"2024-09-22T03:36:01.714468+0000","last_clean":"2024-09-22T03:36:01.714468+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:36:01.714468+0000","last_undegraded":"2024-09-22T03:36:01.714468+0000","last_fullsized":"2024-09-22T03:36:01.714468+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"96'111","reported_seq":124,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.546507+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:36:01.546507+0000","last_peered":"2024-09-22T03:36:01.546507+0000","last_clean":"2024-09-22T03:36:01.546507+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:36:01.546507+0000","last_undegraded":"2024-09-22T03:36:01.546507+0000","last_fullsized":"2024-09-22T03:36:01.546507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"96'71","reported_seq":84,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:00.787394+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:36:00.787394+0000","last_peered":"2024-09-22T03:36:00.787394+0000","last_clean":"2024-09-22T03:36:00.787394+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:36:00.787394+0000","last_undegraded":"2024-09-22T03:36:00.787394+0000","last_fullsized":"2024-09-22T03:36:00.787394+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"96'111","reported_seq":124,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.758040+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:36:01.758040+0000","last_peered":"2024-09-22T03:36:01.758040+0000","last_clean":"2024-09-22T03:36:01.758040+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:36:01.758040+0000","last_undegraded":"2024-09-22T03:36:01.758040+0000","last_fullsized":"2024-09-22T03:36:01.758040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"96'71","reported_seq":84,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.266061+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:36:01.266061+0000","last_peered":"2024-09-22T03:36:01.266061+0000","last_clean":"2024-09-22T03:36:01.266061+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:36:01.266061+0000","last_undegraded":"2024-09-22T03:36:01.266061+0000","last_fullsized":"2024-09-22T03:36:01.266061+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"96'135","reported_seq":148,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.110816+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:36:01.110816+0000","last_peered":"2024-09-22T03:36:01.110816+0000","last_clean":"2024-09-22T03:36:01.110816+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:36:01.110816+0000","last_undegraded":"2024-09-22T03:36:01.110816+0000","last_fullsized":"2024-09-22T03:36:01.110816+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":391,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.163695+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:49.163695+0000","last_peered":"2024-09-22T03:35:49.163695+0000","last_clean":"2024-09-22T03:35:49.163695+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:49.163695+0000","last_undegraded":"2024-09-22T03:35:49.163695+0000","last_fullsized":"2024-09-22T03:35:49.163695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":257,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.170082+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:49.170082+0000","last_peered":"2024-09-22T03:35:49.170082+0000","last_clean":"2024-09-22T03:35:49.170082+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:49.170082+0000","last_undegraded":"2024-09-22T03:35:49.170082+0000","last_fullsized":"2024-09-22T03:35:49.170082+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"96'143","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.892798+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:36:01.892798+0000","last_peered":"2024-09-22T03:36:01.892798+0000","last_clean":"2024-09-22T03:36:01.892798+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:36:01.892798+0000","last_undegraded":"2024-09-22T03:36:01.892798+0000","last_fullsized":"2024-09-22T03:36:01.892798+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.169575+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:49.169575+0000","last_peered":"2024-09-22T03:35:49.169575+0000","last_clean":"2024-09-22T03:35:49.169575+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:49.169575+0000","last_undegraded":"2024-09-22T03:35:49.169575+0000","last_fullsized":"2024-09-22T03:35:49.169575+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"96'119","reported_seq":132,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.398616+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:36:01.398616+0000","last_peered":"2024-09-22T03:36:01.398616+0000","last_clean":"2024-09-22T03:36:01.398616+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:36:01.398616+0000","last_undegraded":"2024-09-22T03:36:01.398616+0000","last_fullsized":"2024-09-22T03:36:01.398616+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.165538+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:49.165538+0000","last_peered":"2024-09-22T03:35:49.165538+0000","last_clean":"2024-09-22T03:35:49.165538+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:49.165538+0000","last_undegraded":"2024-09-22T03:35:49.165538+0000","last_fullsized":"2024-09-22T03:35:49.165538+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"96'79","reported_seq":92,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.625944+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:36:01.625944+0000","last_peered":"2024-09-22T03:36:01.625944+0000","last_clean":"2024-09-22T03:36:01.625944+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:36:01.625944+0000","last_undegraded":"2024-09-22T03:36:01.625944+0000","last_fullsized":"2024-09-22T03:36:01.625944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":258,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.165927+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:49.165927+0000","last_peered":"2024-09-22T03:35:49.165927+0000","last_clean":"2024-09-22T03:35:49.165927+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:49.165927+0000","last_undegraded":"2024-09-22T03:35:49.165927+0000","last_fullsized":"2024-09-22T03:35:49.165927+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"96'135","reported_seq":148,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:00.615393+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:36:00.615393+0000","last_peered":"2024-09-22T03:36:00.615393+0000","last_clean":"2024-09-22T03:36:00.615393+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:36:00.615393+0000","last_undegraded":"2024-09-22T03:36:00.615393+0000","last_fullsized":"2024-09-22T03:36:00.615393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.168040+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:49.168040+0000","last_peered":"2024-09-22T03:35:49.168040+0000","last_clean":"2024-09-22T03:35:49.168040+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:49.168040+0000","last_undegraded":"2024-09-22T03:35:49.168040+0000","last_fullsized":"2024-09-22T03:35:49.168040+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"96'103","reported_seq":116,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.962810+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:36:01.962810+0000","last_peered":"2024-09-22T03:36:01.962810+0000","last_clean":"2024-09-22T03:36:01.962810+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:36:01.962810+0000","last_undegraded":"2024-09-22T03:36:01.962810+0000","last_fullsized":"2024-09-22T03:36:01.962810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.169317+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:49.169317+0000","last_peered":"2024-09-22T03:35:49.169317+0000","last_clean":"2024-09-22T03:35:49.169317+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:49.169317+0000","last_undegraded":"2024-09-22T03:35:49.169317+0000","last_fullsized":"2024-09-22T03:35:49.169317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"96'95","reported_seq":108,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:00.938644+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:36:00.938644+0000","last_peered":"2024-09-22T03:36:00.938644+0000","last_clean":"2024-09-22T03:36:00.938644+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:36:00.938644+0000","last_undegraded":"2024-09-22T03:36:00.938644+0000","last_fullsized":"2024-09-22T03:36:00.938644+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.167860+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:49.167860+0000","last_peered":"2024-09-22T03:35:49.167860+0000","last_clean":"2024-09-22T03:35:49.167860+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:49.167860+0000","last_undegraded":"2024-09-22T03:35:49.167860+0000","last_fullsized":"2024-09-22T03:35:49.167860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"96'47","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.152672+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:36:01.152672+0000","last_peered":"2024-09-22T03:36:01.152672+0000","last_clean":"2024-09-22T03:36:01.152672+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:36:01.152672+0000","last_undegraded":"2024-09-22T03:36:01.152672+0000","last_fullsized":"2024-09-22T03:36:01.152672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.172060+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:49.172060+0000","last_peered":"2024-09-22T03:35:49.172060+0000","last_clean":"2024-09-22T03:35:49.172060+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:49.172060+0000","last_undegraded":"2024-09-22T03:35:49.172060+0000","last_fullsized":"2024-09-22T03:35:49.172060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":14376960,"num_objects":220,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":14040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214113304576,"internally_reserved":0,"allocated":98189312,"data_stored":98189312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1739,"ondisk_log_size":1739,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214113304576,"internally_reserved":0,"allocated":98189312,"data_stored":98189312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":19872,"kb_used_data":19872,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52277856,"statfs":{"total":53552873472,"available":53532524544,"internally_reserved":0,"allocated":20348928,"data_stored":20348928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":27004,"kb_used_data":27004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52270724,"statfs":{"total":53552873472,"available":53525221376,"internally_reserved":0,"allocated":27652096,"data_stored":27652096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":21364,"kb_used_data":21364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52276364,"statfs":{"total":53552873472,"available":53530996736,"internally_reserved":0,"allocated":21876736,"data_stored":21876736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23308,"kb_used_data":23308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274420,"statfs":{"total":53552873472,"available":53529006080,"internally_reserved":0,"allocated":23867392,"data_stored":23867392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53527924736,"internally_reserved":0,"allocated":24948736,"data_stored":24948736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53529739264,"internally_reserved":0,"allocated":23134208,"data_stored":23134208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53523976192,"internally_reserved":0,"allocated":28897280,"data_stored":28897280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53527924736,"internally_reserved":0,"allocated":24948736,"data_stored":24948736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53529739264,"internally_reserved":0,"allocated":23134208,"data_stored":23134208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53523976192,"internally_reserved":0,"allocated":28897280,"data_stored":28897280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:36:03.019 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:03.168+0000 7f28d8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:03.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:03.173+0000 7f28d8d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:03.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 14 16 1929 1913 1.06743 1.125 0.153714 0.116356 2024-09-22T03:36:04.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 15 16 2067 2051 1.06814 1.07812 0.15798 0.11621 2024-09-22T03:36:05.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 16 16 2201 2185 1.0668 1.04688 0.06983 0.116902 2024-09-22T03:36:05.310 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:05.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:05.422+0000 7f208a223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:05.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:05.429+0000 7f208a223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:06.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 17 16 2318 2302 1.05781 0.914062 0.033776 0.117511 2024-09-22T03:36:07.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 18 16 2449 2433 1.0559 1.02344 0.0322225 0.117936 2024-09-22T03:36:07.378 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:07.378 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:07.379 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:36:07.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:07.494+0000 7f7a78797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:07.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:07.502+0000 7f7a78797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:07.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:07.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:07.778+0000 7f15d2458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:07.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:07.785+0000 7f15d2458640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:07.840 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:36:07.840 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:36:07.859 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-22T03:36:06.080737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":14967347,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":1894,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1839,"ondisk_log_size":1839,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":91548,"kb_used_data":91548,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209099364,"statfs":{"total":214211493888,"available":214117748736,"internally_reserved":0,"allocated":93745152,"data_stored":93745152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11362304,"num_objects":174,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1387,"num_write_kb":11096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002133"},"pg_stats":[{"pgid":"10.7","version":"96'98","reported_seq":111,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.992496+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:36:01.992496+0000","last_peered":"2024-09-22T03:36:01.992496+0000","last_clean":"2024-09-22T03:36:01.992496+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:36:01.992496+0000","last_undegraded":"2024-09-22T03:36:01.992496+0000","last_fullsized":"2024-09-22T03:36:01.992496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":811008,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"96'135","reported_seq":148,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.485839+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:36:01.485839+0000","last_peered":"2024-09-22T03:36:01.485839+0000","last_clean":"2024-09-22T03:36:01.485839+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:36:01.485839+0000","last_undegraded":"2024-09-22T03:36:01.485839+0000","last_fullsized":"2024-09-22T03:36:01.485839+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"96'143","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.458318+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:36:01.458318+0000","last_peered":"2024-09-22T03:36:01.458318+0000","last_clean":"2024-09-22T03:36:01.458318+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:36:01.458318+0000","last_undegraded":"2024-09-22T03:36:01.458318+0000","last_fullsized":"2024-09-22T03:36:01.458318+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"96'143","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.714468+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:36:01.714468+0000","last_peered":"2024-09-22T03:36:01.714468+0000","last_clean":"2024-09-22T03:36:01.714468+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:36:01.714468+0000","last_undegraded":"2024-09-22T03:36:01.714468+0000","last_fullsized":"2024-09-22T03:36:01.714468+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"96'111","reported_seq":124,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.546507+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:36:01.546507+0000","last_peered":"2024-09-22T03:36:01.546507+0000","last_clean":"2024-09-22T03:36:01.546507+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:36:01.546507+0000","last_undegraded":"2024-09-22T03:36:01.546507+0000","last_fullsized":"2024-09-22T03:36:01.546507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"96'71","reported_seq":84,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:00.787394+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:36:00.787394+0000","last_peered":"2024-09-22T03:36:00.787394+0000","last_clean":"2024-09-22T03:36:00.787394+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:36:00.787394+0000","last_undegraded":"2024-09-22T03:36:00.787394+0000","last_fullsized":"2024-09-22T03:36:00.787394+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"96'111","reported_seq":124,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.758040+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:36:01.758040+0000","last_peered":"2024-09-22T03:36:01.758040+0000","last_clean":"2024-09-22T03:36:01.758040+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:36:01.758040+0000","last_undegraded":"2024-09-22T03:36:01.758040+0000","last_fullsized":"2024-09-22T03:36:01.758040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"96'71","reported_seq":84,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.266061+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:36:01.266061+0000","last_peered":"2024-09-22T03:36:01.266061+0000","last_clean":"2024-09-22T03:36:01.266061+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:36:01.266061+0000","last_undegraded":"2024-09-22T03:36:01.266061+0000","last_fullsized":"2024-09-22T03:36:01.266061+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"96'135","reported_seq":148,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.110816+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:36:01.110816+0000","last_peered":"2024-09-22T03:36:01.110816+0000","last_clean":"2024-09-22T03:36:01.110816+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:36:01.110816+0000","last_undegraded":"2024-09-22T03:36:01.110816+0000","last_fullsized":"2024-09-22T03:36:01.110816+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":391,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.163695+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:35:49.163695+0000","last_peered":"2024-09-22T03:35:49.163695+0000","last_clean":"2024-09-22T03:35:49.163695+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:35:49.163695+0000","last_undegraded":"2024-09-22T03:35:49.163695+0000","last_fullsized":"2024-09-22T03:35:49.163695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":257,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.170082+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:35:49.170082+0000","last_peered":"2024-09-22T03:35:49.170082+0000","last_clean":"2024-09-22T03:35:49.170082+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:35:49.170082+0000","last_undegraded":"2024-09-22T03:35:49.170082+0000","last_fullsized":"2024-09-22T03:35:49.170082+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"96'143","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.892798+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:36:01.892798+0000","last_peered":"2024-09-22T03:36:01.892798+0000","last_clean":"2024-09-22T03:36:01.892798+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:36:01.892798+0000","last_undegraded":"2024-09-22T03:36:01.892798+0000","last_fullsized":"2024-09-22T03:36:01.892798+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.169575+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:35:49.169575+0000","last_peered":"2024-09-22T03:35:49.169575+0000","last_clean":"2024-09-22T03:35:49.169575+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:35:49.169575+0000","last_undegraded":"2024-09-22T03:35:49.169575+0000","last_fullsized":"2024-09-22T03:35:49.169575+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"96'119","reported_seq":132,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.398616+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:36:01.398616+0000","last_peered":"2024-09-22T03:36:01.398616+0000","last_clean":"2024-09-22T03:36:01.398616+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:36:01.398616+0000","last_undegraded":"2024-09-22T03:36:01.398616+0000","last_fullsized":"2024-09-22T03:36:01.398616+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.165538+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:49.165538+0000","last_peered":"2024-09-22T03:35:49.165538+0000","last_clean":"2024-09-22T03:35:49.165538+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:35:49.165538+0000","last_undegraded":"2024-09-22T03:35:49.165538+0000","last_fullsized":"2024-09-22T03:35:49.165538+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"96'79","reported_seq":92,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.625944+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:36:01.625944+0000","last_peered":"2024-09-22T03:36:01.625944+0000","last_clean":"2024-09-22T03:36:01.625944+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:36:01.625944+0000","last_undegraded":"2024-09-22T03:36:01.625944+0000","last_fullsized":"2024-09-22T03:36:01.625944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":258,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.165927+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:35:49.165927+0000","last_peered":"2024-09-22T03:35:49.165927+0000","last_clean":"2024-09-22T03:35:49.165927+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:35:49.165927+0000","last_undegraded":"2024-09-22T03:35:49.165927+0000","last_fullsized":"2024-09-22T03:35:49.165927+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"96'135","reported_seq":148,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:00.615393+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:36:00.615393+0000","last_peered":"2024-09-22T03:36:00.615393+0000","last_clean":"2024-09-22T03:36:00.615393+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:36:00.615393+0000","last_undegraded":"2024-09-22T03:36:00.615393+0000","last_fullsized":"2024-09-22T03:36:00.615393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.168040+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:49.168040+0000","last_peered":"2024-09-22T03:35:49.168040+0000","last_clean":"2024-09-22T03:35:49.168040+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:35:49.168040+0000","last_undegraded":"2024-09-22T03:35:49.168040+0000","last_fullsized":"2024-09-22T03:35:49.168040+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"96'103","reported_seq":116,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.962810+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:36:01.962810+0000","last_peered":"2024-09-22T03:36:01.962810+0000","last_clean":"2024-09-22T03:36:01.962810+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:36:01.962810+0000","last_undegraded":"2024-09-22T03:36:01.962810+0000","last_fullsized":"2024-09-22T03:36:01.962810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.169317+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:35:49.169317+0000","last_peered":"2024-09-22T03:35:49.169317+0000","last_clean":"2024-09-22T03:35:49.169317+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:35:49.169317+0000","last_undegraded":"2024-09-22T03:35:49.169317+0000","last_fullsized":"2024-09-22T03:35:49.169317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"96'95","reported_seq":108,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:00.938644+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:36:00.938644+0000","last_peered":"2024-09-22T03:36:00.938644+0000","last_clean":"2024-09-22T03:36:00.938644+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:36:00.938644+0000","last_undegraded":"2024-09-22T03:36:00.938644+0000","last_fullsized":"2024-09-22T03:36:00.938644+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.167860+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:35:49.167860+0000","last_peered":"2024-09-22T03:35:49.167860+0000","last_clean":"2024-09-22T03:35:49.167860+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:35:49.167860+0000","last_undegraded":"2024-09-22T03:35:49.167860+0000","last_fullsized":"2024-09-22T03:35:49.167860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"96'47","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:36:01.152672+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:36:01.152672+0000","last_peered":"2024-09-22T03:36:01.152672+0000","last_clean":"2024-09-22T03:36:01.152672+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:36:01.152672+0000","last_undegraded":"2024-09-22T03:36:01.152672+0000","last_fullsized":"2024-09-22T03:36:01.152672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":256,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-22T03:35:49.172060+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:35:49.172060+0000","last_peered":"2024-09-22T03:35:49.172060+0000","last_clean":"2024-09-22T03:35:49.172060+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:35:49.172060+0000","last_undegraded":"2024-09-22T03:35:49.172060+0000","last_fullsized":"2024-09-22T03:35:49.172060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":14376960,"num_objects":220,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":14040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214113304576,"internally_reserved":0,"allocated":98189312,"data_stored":98189312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1739,"ondisk_log_size":1739,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214113304576,"internally_reserved":0,"allocated":98189312,"data_stored":98189312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":19872,"kb_used_data":19872,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52277856,"statfs":{"total":53552873472,"available":53532524544,"internally_reserved":0,"allocated":20348928,"data_stored":20348928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":27004,"kb_used_data":27004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52270724,"statfs":{"total":53552873472,"available":53525221376,"internally_reserved":0,"allocated":27652096,"data_stored":27652096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":21364,"kb_used_data":21364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52276364,"statfs":{"total":53552873472,"available":53530996736,"internally_reserved":0,"allocated":21876736,"data_stored":21876736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":23308,"kb_used_data":23308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52274420,"statfs":{"total":53552873472,"available":53529006080,"internally_reserved":0,"allocated":23867392,"data_stored":23867392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53527924736,"internally_reserved":0,"allocated":24948736,"data_stored":24948736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53529739264,"internally_reserved":0,"allocated":23134208,"data_stored":23134208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53523976192,"internally_reserved":0,"allocated":28897280,"data_stored":28897280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53527924736,"internally_reserved":0,"allocated":24948736,"data_stored":24948736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53529739264,"internally_reserved":0,"allocated":23134208,"data_stored":23134208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53523976192,"internally_reserved":0,"allocated":28897280,"data_stored":28897280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53531664384,"internally_reserved":0,"allocated":21209088,"data_stored":21209088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:36:08.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 19 16 2583 2567 1.05542 1.04688 0.120087 0.117899 2024-09-22T03:36:09.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:36:09.228283+0000 min lat: 0.0145288 max lat: 0.54404 avg lat: 0.117817 2024-09-22T03:36:09.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:36:09.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 20 16 2726 2710 1.0585 1.11719 0.0912765 0.117817 2024-09-22T03:36:10.033 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:10.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:10.149+0000 7f25ef197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:10.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:10.154+0000 7f25ef197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:10.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 21 16 2851 2835 1.0546 0.976562 0.219336 0.117974 2024-09-22T03:36:11.228 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 22 16 2997 2981 1.0585 1.14062 0.168594 0.117639 2024-09-22T03:36:12.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 23 16 3156 3140 1.06648 1.24219 0.0372734 0.116894 2024-09-22T03:36:12.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:12.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:12.373+0000 7f7071157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:12.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:12.378+0000 7f7071157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:12.861 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:12.861 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:12.861 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:36:12.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:36:12.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:12.976+0000 7fe7b766c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:12.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:12.984+0000 7fe7b766c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:13.010 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:13.011 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:13.013 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:13.014 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:13.014 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:36:13.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:13.137+0000 7f4b5e357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:13.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:13.150+0000 7f4b5e357640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:13.199 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:13.203 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:13.203 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:13.203 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:36:13.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 24 16 3287 3271 1.06468 1.02344 0.0737744 0.116978 2024-09-22T03:36:13.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:13.355+0000 7f3b2b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:13.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:13.364+0000 7f3b2b397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:13.384 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:13.385 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:13.388 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:13.388 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:13.388 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:36:13.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:13.555+0000 7f1afbe2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:13.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:13.564+0000 7f1afbe2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:13.579 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:13.579 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:13.580 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:13.583 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:13.583 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:14.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 25 16 3428 3412 1.06616 1.10156 0.0867989 0.117053 2024-09-22T03:36:14.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:14.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:14.691+0000 7f9c0d62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:14.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:14.698+0000 7f9c0d62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:15.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 26 16 3553 3537 1.06271 0.976562 0.0720333 0.117439 2024-09-22T03:36:16.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 27 16 3686 3670 1.06183 1.03906 0.214453 0.117493 2024-09-22T03:36:17.000 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:17.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:17.135+0000 7fcd77797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:17.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:17.141+0000 7fcd77797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:17.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 28 16 3822 3806 1.06185 1.0625 0.112383 0.11756 2024-09-22T03:36:18.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 29 16 3962 3946 1.06294 1.09375 0.1115 0.117207 2024-09-22T03:36:18.584 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:18.584 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:18.585 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:36:18.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:36:18.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:18.713+0000 7fbd5af0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:18.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:18.722+0000 7fbd5af0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:18.799 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:18.799 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:18.799 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:18.799 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:18.799 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:18.799 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:18.800 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:18.800 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:18.800 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:18.800 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:18.800 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:18.800 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:18.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:18.803 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:18.804 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:36:18.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:18.951+0000 7f83b866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:18.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:18.964+0000 7f83b866c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:19.017 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:19.018 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:19.021 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:19.021 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:19.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:36:19.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:19.174+0000 7fbfa1a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:19.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:19.184+0000 7fbfa1a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:19.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 30 16 4103 4087 1.06423 1.10156 0.0407652 0.11716 2024-09-22T03:36:19.257 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:19.257 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:19.257 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:19.257 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:19.257 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:19.261 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:19.261 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:19.261 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:36:19.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:19.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:19.412+0000 7f1151517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:19.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:19.428+0000 7f1151517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:19.451 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:19.455 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:19.455 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:19.477+0000 7f6bc3f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:19.485+0000 7f6bc3f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:20.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 31 16 4250 4234 1.06694 1.14844 0.0937614 0.116932 2024-09-22T03:36:21.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 32 15 4403 4388 1.07119 1.20312 0.0678563 0.116472 2024-09-22T03:36:21.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:21.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:21.836+0000 7f10e45fe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:21.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:21.841+0000 7f10e45fe640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:22.229 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 33 16 4531 4515 1.06879 0.992188 0.0533538 0.116617 2024-09-22T03:36:23.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 34 16 4654 4638 1.06562 0.960938 0.196822 0.117045 2024-09-22T03:36:24.027 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:24.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.140+0000 7fba28a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:24.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.145+0000 7fba28a2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:24.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 35 16 4787 4771 1.06486 1.03906 0.0552958 0.117126 2024-09-22T03:36:24.456 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:24.456 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:24.457 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:36:24.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:36:24.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.588+0000 7fa45e4fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:24.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.605+0000 7fa45e4fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:24.683 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:24.684 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:24.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:24.686 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:24.687 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:36:24.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.806+0000 7fa52cd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:24.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.817+0000 7fa52cd97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:24.836 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:24.836 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:24.837 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:24.839 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:24.840 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:24.840 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:36:24.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.973+0000 7f2a6c22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:24.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:24.986+0000 7f2a6c22c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:25.028 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:25.028 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:25.028 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:25.028 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:25.028 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:25.028 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:25.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:25.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:25.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:25.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:25.029 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:25.029 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:25.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:25.031 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:25.032 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:36:25.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:25.142+0000 7f8922757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:25.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:25.151+0000 7f8922757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:25.166 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:25.166 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:25.166 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:25.166 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:25.167 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:25.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:25.170 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:25.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 36 16 4893 4877 1.05828 0.828125 0.197829 0.117591 2024-09-22T03:36:26.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 37 16 5062 5046 1.06536 1.32031 0.0640483 0.117133 2024-09-22T03:36:26.459 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:26.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:26.610+0000 7f2c2d66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:26.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:26.618+0000 7f2c2d66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:27.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 38 16 5200 5184 1.06569 1.07812 0.157738 0.11703 2024-09-22T03:36:28.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 39 16 5338 5322 1.06601 1.07812 0.228548 0.116916 2024-09-22T03:36:28.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:28.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:28.900+0000 7fa4a4e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:28.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:28.905+0000 7fa4a4e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:29.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:36:29.230225+0000 min lat: 0.0145288 max lat: 0.54404 avg lat: 0.117628 2024-09-22T03:36:29.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:36:29.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 40 16 5441 5425 1.05947 0.804688 0.0423402 0.117628 2024-09-22T03:36:30.171 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:30.171 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:36:30.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 41 16 5573 5557 1.05878 1.03125 0.0609326 0.117865 2024-09-22T03:36:30.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:30.318+0000 7f31aba6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:30.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:30.337+0000 7f31aba6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:30.825 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:36:30.851 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:36:31.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:31.010+0000 7fa01a42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:31.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:31.023+0000 7fa01a42c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:31.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:31.223+0000 7f085bc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:31.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 42 16 5708 5692 1.05868 1.05469 0.0261302 0.117665 2024-09-22T03:36:31.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:31.229+0000 7f085bc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:31.826 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:36:31.847 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:36:31.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:31.972+0000 7f299a517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:31.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:31.984+0000 7f299a517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:32.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 43 16 5835 5819 1.05713 0.992188 0.0894076 0.117965 2024-09-22T03:36:32.830 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:36:32.854 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:36:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:32.968+0000 7fe09d397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:32.978+0000 7fe09d397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:33.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 44 16 5992 5976 1.06098 1.22656 0.125382 0.11768 2024-09-22T03:36:33.469 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:33.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:33.595+0000 7f04de62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:33.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:33.600+0000 7f04de62c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:33.835 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:36:33.854 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:36:33.854 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:36:34.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:33.997+0000 7f8e4a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:34.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:34.008+0000 7f8e4a197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:34.229 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:36:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:36:34.230 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 45 16 6132 6116 1.06171 1.09375 0.182418 0.11754 2024-09-22T03:36:34.242 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-22T03:36:34.085998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48161331,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5946,"num_write_kb":48352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5904,"ondisk_log_size":5904,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":292536,"kb_used_data":292536,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208898376,"statfs":{"total":214211493888,"available":213911937024,"internally_reserved":0,"allocated":299556864,"data_stored":299556864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5226496,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":638,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000694"},"pg_stats":[{"pgid":"10.7","version":"97'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.906217+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:36:31.906217+0000","last_peered":"2024-09-22T03:36:31.906217+0000","last_clean":"2024-09-22T03:36:31.906217+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:36:31.906217+0000","last_undegraded":"2024-09-22T03:36:31.906217+0000","last_fullsized":"2024-09-22T03:36:31.906217+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"96'408","reported_seq":426,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.854017+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:36:31.854017+0000","last_peered":"2024-09-22T03:36:31.854017+0000","last_clean":"2024-09-22T03:36:31.854017+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:36:31.854017+0000","last_undegraded":"2024-09-22T03:36:31.854017+0000","last_fullsized":"2024-09-22T03:36:31.854017+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"96'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.865297+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:36:31.865297+0000","last_peered":"2024-09-22T03:36:31.865297+0000","last_clean":"2024-09-22T03:36:31.865297+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:36:31.865297+0000","last_undegraded":"2024-09-22T03:36:31.865297+0000","last_fullsized":"2024-09-22T03:36:31.865297+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"97'352","reported_seq":370,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.874313+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:36:31.874313+0000","last_peered":"2024-09-22T03:36:31.874313+0000","last_clean":"2024-09-22T03:36:31.874313+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:36:31.874313+0000","last_undegraded":"2024-09-22T03:36:31.874313+0000","last_fullsized":"2024-09-22T03:36:31.874313+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"96'344","reported_seq":362,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.914052+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:36:31.914052+0000","last_peered":"2024-09-22T03:36:31.914052+0000","last_clean":"2024-09-22T03:36:31.914052+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:36:31.914052+0000","last_undegraded":"2024-09-22T03:36:31.914052+0000","last_fullsized":"2024-09-22T03:36:31.914052+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"97'336","reported_seq":354,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.877557+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:36:31.877557+0000","last_peered":"2024-09-22T03:36:31.877557+0000","last_clean":"2024-09-22T03:36:31.877557+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:36:31.877557+0000","last_undegraded":"2024-09-22T03:36:31.877557+0000","last_fullsized":"2024-09-22T03:36:31.877557+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"96'320","reported_seq":338,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.908545+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:36:31.908545+0000","last_peered":"2024-09-22T03:36:31.908545+0000","last_clean":"2024-09-22T03:36:31.908545+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:36:31.908545+0000","last_undegraded":"2024-09-22T03:36:31.908545+0000","last_fullsized":"2024-09-22T03:36:31.908545+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"97'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.854655+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:36:31.854655+0000","last_peered":"2024-09-22T03:36:31.854655+0000","last_clean":"2024-09-22T03:36:31.854655+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:36:31.854655+0000","last_undegraded":"2024-09-22T03:36:31.854655+0000","last_fullsized":"2024-09-22T03:36:31.854655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"96'312","reported_seq":330,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.855077+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:36:31.855077+0000","last_peered":"2024-09-22T03:36:31.855077+0000","last_clean":"2024-09-22T03:36:31.855077+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:36:31.855077+0000","last_undegraded":"2024-09-22T03:36:31.855077+0000","last_fullsized":"2024-09-22T03:36:31.855077+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":397,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.873971+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:36:31.873971+0000","last_peered":"2024-09-22T03:36:31.873971+0000","last_clean":"2024-09-22T03:36:31.873971+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:36:31.873971+0000","last_undegraded":"2024-09-22T03:36:31.873971+0000","last_fullsized":"2024-09-22T03:36:31.873971+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":263,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.856165+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:36:31.856165+0000","last_peered":"2024-09-22T03:36:31.856165+0000","last_clean":"2024-09-22T03:36:31.856165+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:36:31.856165+0000","last_undegraded":"2024-09-22T03:36:31.856165+0000","last_fullsized":"2024-09-22T03:36:31.856165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"97'408","reported_seq":426,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.867803+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:36:31.867803+0000","last_peered":"2024-09-22T03:36:31.867803+0000","last_clean":"2024-09-22T03:36:31.867803+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:36:31.867803+0000","last_undegraded":"2024-09-22T03:36:31.867803+0000","last_fullsized":"2024-09-22T03:36:31.867803+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.913017+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:36:31.913017+0000","last_peered":"2024-09-22T03:36:31.913017+0000","last_clean":"2024-09-22T03:36:31.913017+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:36:31.913017+0000","last_undegraded":"2024-09-22T03:36:31.913017+0000","last_fullsized":"2024-09-22T03:36:31.913017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"97'464","reported_seq":482,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.876464+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:36:31.876464+0000","last_peered":"2024-09-22T03:36:31.876464+0000","last_clean":"2024-09-22T03:36:31.876464+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:36:31.876464+0000","last_undegraded":"2024-09-22T03:36:31.876464+0000","last_fullsized":"2024-09-22T03:36:31.876464+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.851085+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:31.851085+0000","last_peered":"2024-09-22T03:36:31.851085+0000","last_clean":"2024-09-22T03:36:31.851085+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:36:31.851085+0000","last_undegraded":"2024-09-22T03:36:31.851085+0000","last_fullsized":"2024-09-22T03:36:31.851085+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"97'368","reported_seq":386,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.912213+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:36:31.912213+0000","last_peered":"2024-09-22T03:36:31.912213+0000","last_clean":"2024-09-22T03:36:31.912213+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:36:31.912213+0000","last_undegraded":"2024-09-22T03:36:31.912213+0000","last_fullsized":"2024-09-22T03:36:31.912213+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.906815+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:31.906815+0000","last_peered":"2024-09-22T03:36:31.906815+0000","last_clean":"2024-09-22T03:36:31.906815+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:36:31.906815+0000","last_undegraded":"2024-09-22T03:36:31.906815+0000","last_fullsized":"2024-09-22T03:36:31.906815+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"98'414","reported_seq":433,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.995891+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:36:31.995891+0000","last_peered":"2024-09-22T03:36:31.995891+0000","last_clean":"2024-09-22T03:36:31.995891+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:36:31.995891+0000","last_undegraded":"2024-09-22T03:36:31.995891+0000","last_fullsized":"2024-09-22T03:36:31.995891+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3399680,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.866459+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:31.866459+0000","last_peered":"2024-09-22T03:36:31.866459+0000","last_clean":"2024-09-22T03:36:31.866459+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:36:31.866459+0000","last_undegraded":"2024-09-22T03:36:31.866459+0000","last_fullsized":"2024-09-22T03:36:31.866459+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"96'312","reported_seq":330,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.913498+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:36:31.913498+0000","last_peered":"2024-09-22T03:36:31.913498+0000","last_clean":"2024-09-22T03:36:31.913498+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:36:31.913498+0000","last_undegraded":"2024-09-22T03:36:31.913498+0000","last_fullsized":"2024-09-22T03:36:31.913498+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.876714+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:31.876714+0000","last_peered":"2024-09-22T03:36:31.876714+0000","last_clean":"2024-09-22T03:36:31.876714+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:36:31.876714+0000","last_undegraded":"2024-09-22T03:36:31.876714+0000","last_fullsized":"2024-09-22T03:36:31.876714+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"98'335","reported_seq":354,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.918392+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:36:31.918392+0000","last_peered":"2024-09-22T03:36:31.918392+0000","last_clean":"2024-09-22T03:36:31.918392+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:36:31.918392+0000","last_undegraded":"2024-09-22T03:36:31.918392+0000","last_fullsized":"2024-09-22T03:36:31.918392+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.867376+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:36:31.867376+0000","last_peered":"2024-09-22T03:36:31.867376+0000","last_clean":"2024-09-22T03:36:31.867376+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:36:31.867376+0000","last_undegraded":"2024-09-22T03:36:31.867376+0000","last_fullsized":"2024-09-22T03:36:31.867376+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"98'303","reported_seq":322,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.874536+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:36:31.874536+0000","last_peered":"2024-09-22T03:36:31.874536+0000","last_clean":"2024-09-22T03:36:31.874536+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:36:31.874536+0000","last_undegraded":"2024-09-22T03:36:31.874536+0000","last_fullsized":"2024-09-22T03:36:31.874536+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.853187+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:36:31.853187+0000","last_peered":"2024-09-22T03:36:31.853187+0000","last_clean":"2024-09-22T03:36:31.853187+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:36:31.853187+0000","last_undegraded":"2024-09-22T03:36:31.853187+0000","last_fullsized":"2024-09-22T03:36:31.853187+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":47570944,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5807,"num_write_kb":46456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213907263488,"internally_reserved":0,"allocated":304230400,"data_stored":304230400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5804,"ondisk_log_size":5804,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213907345408,"internally_reserved":0,"allocated":304148480,"data_stored":304148480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":57232,"kb_used_data":57232,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52240496,"statfs":{"total":53552873472,"available":53494267904,"internally_reserved":0,"allocated":58605568,"data_stored":58605568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":84264,"kb_used_data":84264,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52213464,"statfs":{"total":53552873472,"available":53466587136,"internally_reserved":0,"allocated":86286336,"data_stored":86286336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":71012,"kb_used_data":71012,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52226716,"statfs":{"total":53552873472,"available":53480157184,"internally_reserved":0,"allocated":72716288,"data_stored":72716288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":80028,"kb_used_data":80028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52217700,"statfs":{"total":53552873472,"available":53470924800,"internally_reserved":0,"allocated":81948672,"data_stored":81948672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53469618176,"internally_reserved":0,"allocated":83255296,"data_stored":83255296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53479116800,"internally_reserved":0,"allocated":73756672,"data_stored":73756672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53465403392,"internally_reserved":0,"allocated":87470080,"data_stored":87470080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53469618176,"internally_reserved":0,"allocated":83255296,"data_stored":83255296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53479043072,"internally_reserved":0,"allocated":73830400,"data_stored":73830400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53465395200,"internally_reserved":0,"allocated":87478272,"data_stored":87478272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:36:34.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:36:34.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:34.371+0000 7f24c1d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:34.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:34.379+0000 7f24c1d0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:34.576 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:36:34.577 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:36:34.590 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-22T03:36:34.085998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48161331,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5946,"num_write_kb":48352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5904,"ondisk_log_size":5904,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":292536,"kb_used_data":292536,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208898376,"statfs":{"total":214211493888,"available":213911937024,"internally_reserved":0,"allocated":299556864,"data_stored":299556864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5226496,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":638,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000694"},"pg_stats":[{"pgid":"10.7","version":"97'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.906217+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:36:31.906217+0000","last_peered":"2024-09-22T03:36:31.906217+0000","last_clean":"2024-09-22T03:36:31.906217+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:36:31.906217+0000","last_undegraded":"2024-09-22T03:36:31.906217+0000","last_fullsized":"2024-09-22T03:36:31.906217+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"96'408","reported_seq":426,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.854017+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:36:31.854017+0000","last_peered":"2024-09-22T03:36:31.854017+0000","last_clean":"2024-09-22T03:36:31.854017+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:36:31.854017+0000","last_undegraded":"2024-09-22T03:36:31.854017+0000","last_fullsized":"2024-09-22T03:36:31.854017+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"96'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.865297+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:36:31.865297+0000","last_peered":"2024-09-22T03:36:31.865297+0000","last_clean":"2024-09-22T03:36:31.865297+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:36:31.865297+0000","last_undegraded":"2024-09-22T03:36:31.865297+0000","last_fullsized":"2024-09-22T03:36:31.865297+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"97'352","reported_seq":370,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.874313+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:36:31.874313+0000","last_peered":"2024-09-22T03:36:31.874313+0000","last_clean":"2024-09-22T03:36:31.874313+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:36:31.874313+0000","last_undegraded":"2024-09-22T03:36:31.874313+0000","last_fullsized":"2024-09-22T03:36:31.874313+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"96'344","reported_seq":362,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.914052+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:36:31.914052+0000","last_peered":"2024-09-22T03:36:31.914052+0000","last_clean":"2024-09-22T03:36:31.914052+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:36:31.914052+0000","last_undegraded":"2024-09-22T03:36:31.914052+0000","last_fullsized":"2024-09-22T03:36:31.914052+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"97'336","reported_seq":354,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.877557+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:36:31.877557+0000","last_peered":"2024-09-22T03:36:31.877557+0000","last_clean":"2024-09-22T03:36:31.877557+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:36:31.877557+0000","last_undegraded":"2024-09-22T03:36:31.877557+0000","last_fullsized":"2024-09-22T03:36:31.877557+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"96'320","reported_seq":338,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.908545+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:36:31.908545+0000","last_peered":"2024-09-22T03:36:31.908545+0000","last_clean":"2024-09-22T03:36:31.908545+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:36:31.908545+0000","last_undegraded":"2024-09-22T03:36:31.908545+0000","last_fullsized":"2024-09-22T03:36:31.908545+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"97'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.854655+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:36:31.854655+0000","last_peered":"2024-09-22T03:36:31.854655+0000","last_clean":"2024-09-22T03:36:31.854655+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:36:31.854655+0000","last_undegraded":"2024-09-22T03:36:31.854655+0000","last_fullsized":"2024-09-22T03:36:31.854655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"96'312","reported_seq":330,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.855077+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:36:31.855077+0000","last_peered":"2024-09-22T03:36:31.855077+0000","last_clean":"2024-09-22T03:36:31.855077+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:36:31.855077+0000","last_undegraded":"2024-09-22T03:36:31.855077+0000","last_fullsized":"2024-09-22T03:36:31.855077+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":397,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.873971+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:36:31.873971+0000","last_peered":"2024-09-22T03:36:31.873971+0000","last_clean":"2024-09-22T03:36:31.873971+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:36:31.873971+0000","last_undegraded":"2024-09-22T03:36:31.873971+0000","last_fullsized":"2024-09-22T03:36:31.873971+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":263,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.856165+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:36:31.856165+0000","last_peered":"2024-09-22T03:36:31.856165+0000","last_clean":"2024-09-22T03:36:31.856165+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:36:31.856165+0000","last_undegraded":"2024-09-22T03:36:31.856165+0000","last_fullsized":"2024-09-22T03:36:31.856165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"97'408","reported_seq":426,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.867803+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:36:31.867803+0000","last_peered":"2024-09-22T03:36:31.867803+0000","last_clean":"2024-09-22T03:36:31.867803+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:36:31.867803+0000","last_undegraded":"2024-09-22T03:36:31.867803+0000","last_fullsized":"2024-09-22T03:36:31.867803+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.913017+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:36:31.913017+0000","last_peered":"2024-09-22T03:36:31.913017+0000","last_clean":"2024-09-22T03:36:31.913017+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:36:31.913017+0000","last_undegraded":"2024-09-22T03:36:31.913017+0000","last_fullsized":"2024-09-22T03:36:31.913017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"97'464","reported_seq":482,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.876464+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:36:31.876464+0000","last_peered":"2024-09-22T03:36:31.876464+0000","last_clean":"2024-09-22T03:36:31.876464+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:36:31.876464+0000","last_undegraded":"2024-09-22T03:36:31.876464+0000","last_fullsized":"2024-09-22T03:36:31.876464+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.851085+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:31.851085+0000","last_peered":"2024-09-22T03:36:31.851085+0000","last_clean":"2024-09-22T03:36:31.851085+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:36:31.851085+0000","last_undegraded":"2024-09-22T03:36:31.851085+0000","last_fullsized":"2024-09-22T03:36:31.851085+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"97'368","reported_seq":386,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.912213+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:36:31.912213+0000","last_peered":"2024-09-22T03:36:31.912213+0000","last_clean":"2024-09-22T03:36:31.912213+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:36:31.912213+0000","last_undegraded":"2024-09-22T03:36:31.912213+0000","last_fullsized":"2024-09-22T03:36:31.912213+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.906815+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:31.906815+0000","last_peered":"2024-09-22T03:36:31.906815+0000","last_clean":"2024-09-22T03:36:31.906815+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:36:31.906815+0000","last_undegraded":"2024-09-22T03:36:31.906815+0000","last_fullsized":"2024-09-22T03:36:31.906815+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"98'414","reported_seq":433,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.995891+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:36:31.995891+0000","last_peered":"2024-09-22T03:36:31.995891+0000","last_clean":"2024-09-22T03:36:31.995891+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:36:31.995891+0000","last_undegraded":"2024-09-22T03:36:31.995891+0000","last_fullsized":"2024-09-22T03:36:31.995891+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3399680,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.866459+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:31.866459+0000","last_peered":"2024-09-22T03:36:31.866459+0000","last_clean":"2024-09-22T03:36:31.866459+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:36:31.866459+0000","last_undegraded":"2024-09-22T03:36:31.866459+0000","last_fullsized":"2024-09-22T03:36:31.866459+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"96'312","reported_seq":330,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.913498+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:36:31.913498+0000","last_peered":"2024-09-22T03:36:31.913498+0000","last_clean":"2024-09-22T03:36:31.913498+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:36:31.913498+0000","last_undegraded":"2024-09-22T03:36:31.913498+0000","last_fullsized":"2024-09-22T03:36:31.913498+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.876714+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:31.876714+0000","last_peered":"2024-09-22T03:36:31.876714+0000","last_clean":"2024-09-22T03:36:31.876714+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:36:31.876714+0000","last_undegraded":"2024-09-22T03:36:31.876714+0000","last_fullsized":"2024-09-22T03:36:31.876714+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"98'335","reported_seq":354,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.918392+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:36:31.918392+0000","last_peered":"2024-09-22T03:36:31.918392+0000","last_clean":"2024-09-22T03:36:31.918392+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:36:31.918392+0000","last_undegraded":"2024-09-22T03:36:31.918392+0000","last_fullsized":"2024-09-22T03:36:31.918392+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.867376+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:36:31.867376+0000","last_peered":"2024-09-22T03:36:31.867376+0000","last_clean":"2024-09-22T03:36:31.867376+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:36:31.867376+0000","last_undegraded":"2024-09-22T03:36:31.867376+0000","last_fullsized":"2024-09-22T03:36:31.867376+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"98'303","reported_seq":322,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.874536+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:36:31.874536+0000","last_peered":"2024-09-22T03:36:31.874536+0000","last_clean":"2024-09-22T03:36:31.874536+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:36:31.874536+0000","last_undegraded":"2024-09-22T03:36:31.874536+0000","last_fullsized":"2024-09-22T03:36:31.874536+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.853187+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:36:31.853187+0000","last_peered":"2024-09-22T03:36:31.853187+0000","last_clean":"2024-09-22T03:36:31.853187+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:36:31.853187+0000","last_undegraded":"2024-09-22T03:36:31.853187+0000","last_fullsized":"2024-09-22T03:36:31.853187+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":47570944,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5807,"num_write_kb":46456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213907263488,"internally_reserved":0,"allocated":304230400,"data_stored":304230400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5804,"ondisk_log_size":5804,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213907345408,"internally_reserved":0,"allocated":304148480,"data_stored":304148480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":57232,"kb_used_data":57232,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52240496,"statfs":{"total":53552873472,"available":53494267904,"internally_reserved":0,"allocated":58605568,"data_stored":58605568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":84264,"kb_used_data":84264,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52213464,"statfs":{"total":53552873472,"available":53466587136,"internally_reserved":0,"allocated":86286336,"data_stored":86286336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":71012,"kb_used_data":71012,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52226716,"statfs":{"total":53552873472,"available":53480157184,"internally_reserved":0,"allocated":72716288,"data_stored":72716288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":80028,"kb_used_data":80028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52217700,"statfs":{"total":53552873472,"available":53470924800,"internally_reserved":0,"allocated":81948672,"data_stored":81948672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53469618176,"internally_reserved":0,"allocated":83255296,"data_stored":83255296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53479116800,"internally_reserved":0,"allocated":73756672,"data_stored":73756672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53465403392,"internally_reserved":0,"allocated":87470080,"data_stored":87470080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53469618176,"internally_reserved":0,"allocated":83255296,"data_stored":83255296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53479043072,"internally_reserved":0,"allocated":73830400,"data_stored":73830400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53465395200,"internally_reserved":0,"allocated":87478272,"data_stored":87478272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:36:34.591 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:36:34.591 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:34.592 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:36:34.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:34.726+0000 7f200e24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:34.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:34.738+0000 7f200e24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:35.091 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:36:35.091 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:36:35.111 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-22T03:36:34.085998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48161331,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":5946,"num_write_kb":48352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5904,"ondisk_log_size":5904,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":292536,"kb_used_data":292536,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208898376,"statfs":{"total":214211493888,"available":213911937024,"internally_reserved":0,"allocated":299556864,"data_stored":299556864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5226496,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":638,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000694"},"pg_stats":[{"pgid":"10.7","version":"97'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.906217+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:36:31.906217+0000","last_peered":"2024-09-22T03:36:31.906217+0000","last_clean":"2024-09-22T03:36:31.906217+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:36:31.906217+0000","last_undegraded":"2024-09-22T03:36:31.906217+0000","last_fullsized":"2024-09-22T03:36:31.906217+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"96'408","reported_seq":426,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.854017+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:36:31.854017+0000","last_peered":"2024-09-22T03:36:31.854017+0000","last_clean":"2024-09-22T03:36:31.854017+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:36:31.854017+0000","last_undegraded":"2024-09-22T03:36:31.854017+0000","last_fullsized":"2024-09-22T03:36:31.854017+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"96'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.865297+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:36:31.865297+0000","last_peered":"2024-09-22T03:36:31.865297+0000","last_clean":"2024-09-22T03:36:31.865297+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:36:31.865297+0000","last_undegraded":"2024-09-22T03:36:31.865297+0000","last_fullsized":"2024-09-22T03:36:31.865297+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"97'352","reported_seq":370,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.874313+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:36:31.874313+0000","last_peered":"2024-09-22T03:36:31.874313+0000","last_clean":"2024-09-22T03:36:31.874313+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:36:31.874313+0000","last_undegraded":"2024-09-22T03:36:31.874313+0000","last_fullsized":"2024-09-22T03:36:31.874313+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"96'344","reported_seq":362,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.914052+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:36:31.914052+0000","last_peered":"2024-09-22T03:36:31.914052+0000","last_clean":"2024-09-22T03:36:31.914052+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:36:31.914052+0000","last_undegraded":"2024-09-22T03:36:31.914052+0000","last_fullsized":"2024-09-22T03:36:31.914052+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"97'336","reported_seq":354,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.877557+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:36:31.877557+0000","last_peered":"2024-09-22T03:36:31.877557+0000","last_clean":"2024-09-22T03:36:31.877557+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:36:31.877557+0000","last_undegraded":"2024-09-22T03:36:31.877557+0000","last_fullsized":"2024-09-22T03:36:31.877557+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"96'320","reported_seq":338,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.908545+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:36:31.908545+0000","last_peered":"2024-09-22T03:36:31.908545+0000","last_clean":"2024-09-22T03:36:31.908545+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:36:31.908545+0000","last_undegraded":"2024-09-22T03:36:31.908545+0000","last_fullsized":"2024-09-22T03:36:31.908545+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"97'376","reported_seq":394,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.854655+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:36:31.854655+0000","last_peered":"2024-09-22T03:36:31.854655+0000","last_clean":"2024-09-22T03:36:31.854655+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:36:31.854655+0000","last_undegraded":"2024-09-22T03:36:31.854655+0000","last_fullsized":"2024-09-22T03:36:31.854655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"96'312","reported_seq":330,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.855077+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:36:31.855077+0000","last_peered":"2024-09-22T03:36:31.855077+0000","last_clean":"2024-09-22T03:36:31.855077+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:36:31.855077+0000","last_undegraded":"2024-09-22T03:36:31.855077+0000","last_fullsized":"2024-09-22T03:36:31.855077+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":397,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.873971+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:36:31.873971+0000","last_peered":"2024-09-22T03:36:31.873971+0000","last_clean":"2024-09-22T03:36:31.873971+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:36:31.873971+0000","last_undegraded":"2024-09-22T03:36:31.873971+0000","last_fullsized":"2024-09-22T03:36:31.873971+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":263,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.856165+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:36:31.856165+0000","last_peered":"2024-09-22T03:36:31.856165+0000","last_clean":"2024-09-22T03:36:31.856165+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:36:31.856165+0000","last_undegraded":"2024-09-22T03:36:31.856165+0000","last_fullsized":"2024-09-22T03:36:31.856165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"97'408","reported_seq":426,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.867803+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:36:31.867803+0000","last_peered":"2024-09-22T03:36:31.867803+0000","last_clean":"2024-09-22T03:36:31.867803+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:36:31.867803+0000","last_undegraded":"2024-09-22T03:36:31.867803+0000","last_fullsized":"2024-09-22T03:36:31.867803+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.913017+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:36:31.913017+0000","last_peered":"2024-09-22T03:36:31.913017+0000","last_clean":"2024-09-22T03:36:31.913017+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:36:31.913017+0000","last_undegraded":"2024-09-22T03:36:31.913017+0000","last_fullsized":"2024-09-22T03:36:31.913017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"97'464","reported_seq":482,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.876464+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:36:31.876464+0000","last_peered":"2024-09-22T03:36:31.876464+0000","last_clean":"2024-09-22T03:36:31.876464+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:36:31.876464+0000","last_undegraded":"2024-09-22T03:36:31.876464+0000","last_fullsized":"2024-09-22T03:36:31.876464+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.851085+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:31.851085+0000","last_peered":"2024-09-22T03:36:31.851085+0000","last_clean":"2024-09-22T03:36:31.851085+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:36:31.851085+0000","last_undegraded":"2024-09-22T03:36:31.851085+0000","last_fullsized":"2024-09-22T03:36:31.851085+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"97'368","reported_seq":386,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.912213+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:36:31.912213+0000","last_peered":"2024-09-22T03:36:31.912213+0000","last_clean":"2024-09-22T03:36:31.912213+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:36:31.912213+0000","last_undegraded":"2024-09-22T03:36:31.912213+0000","last_fullsized":"2024-09-22T03:36:31.912213+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.906815+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:31.906815+0000","last_peered":"2024-09-22T03:36:31.906815+0000","last_clean":"2024-09-22T03:36:31.906815+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:36:31.906815+0000","last_undegraded":"2024-09-22T03:36:31.906815+0000","last_fullsized":"2024-09-22T03:36:31.906815+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"98'414","reported_seq":433,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.995891+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:36:31.995891+0000","last_peered":"2024-09-22T03:36:31.995891+0000","last_clean":"2024-09-22T03:36:31.995891+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:36:31.995891+0000","last_undegraded":"2024-09-22T03:36:31.995891+0000","last_fullsized":"2024-09-22T03:36:31.995891+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3399680,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.866459+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:31.866459+0000","last_peered":"2024-09-22T03:36:31.866459+0000","last_clean":"2024-09-22T03:36:31.866459+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:36:31.866459+0000","last_undegraded":"2024-09-22T03:36:31.866459+0000","last_fullsized":"2024-09-22T03:36:31.866459+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"96'312","reported_seq":330,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.913498+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:36:31.913498+0000","last_peered":"2024-09-22T03:36:31.913498+0000","last_clean":"2024-09-22T03:36:31.913498+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:36:31.913498+0000","last_undegraded":"2024-09-22T03:36:31.913498+0000","last_fullsized":"2024-09-22T03:36:31.913498+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.876714+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:31.876714+0000","last_peered":"2024-09-22T03:36:31.876714+0000","last_clean":"2024-09-22T03:36:31.876714+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:36:31.876714+0000","last_undegraded":"2024-09-22T03:36:31.876714+0000","last_fullsized":"2024-09-22T03:36:31.876714+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"98'335","reported_seq":354,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.918392+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:36:31.918392+0000","last_peered":"2024-09-22T03:36:31.918392+0000","last_clean":"2024-09-22T03:36:31.918392+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:36:31.918392+0000","last_undegraded":"2024-09-22T03:36:31.918392+0000","last_fullsized":"2024-09-22T03:36:31.918392+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.867376+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:36:31.867376+0000","last_peered":"2024-09-22T03:36:31.867376+0000","last_clean":"2024-09-22T03:36:31.867376+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:36:31.867376+0000","last_undegraded":"2024-09-22T03:36:31.867376+0000","last_fullsized":"2024-09-22T03:36:31.867376+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"98'303","reported_seq":322,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.874536+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:36:31.874536+0000","last_peered":"2024-09-22T03:36:31.874536+0000","last_clean":"2024-09-22T03:36:31.874536+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:36:31.874536+0000","last_undegraded":"2024-09-22T03:36:31.874536+0000","last_fullsized":"2024-09-22T03:36:31.874536+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":262,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-22T03:36:31.853187+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:36:31.853187+0000","last_peered":"2024-09-22T03:36:31.853187+0000","last_clean":"2024-09-22T03:36:31.853187+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:36:31.853187+0000","last_undegraded":"2024-09-22T03:36:31.853187+0000","last_fullsized":"2024-09-22T03:36:31.853187+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":47570944,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5807,"num_write_kb":46456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213907263488,"internally_reserved":0,"allocated":304230400,"data_stored":304230400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5804,"ondisk_log_size":5804,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213907345408,"internally_reserved":0,"allocated":304148480,"data_stored":304148480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":57232,"kb_used_data":57232,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52240496,"statfs":{"total":53552873472,"available":53494267904,"internally_reserved":0,"allocated":58605568,"data_stored":58605568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":84264,"kb_used_data":84264,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52213464,"statfs":{"total":53552873472,"available":53466587136,"internally_reserved":0,"allocated":86286336,"data_stored":86286336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":71012,"kb_used_data":71012,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52226716,"statfs":{"total":53552873472,"available":53480157184,"internally_reserved":0,"allocated":72716288,"data_stored":72716288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":80028,"kb_used_data":80028,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52217700,"statfs":{"total":53552873472,"available":53470924800,"internally_reserved":0,"allocated":81948672,"data_stored":81948672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53469618176,"internally_reserved":0,"allocated":83255296,"data_stored":83255296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53479116800,"internally_reserved":0,"allocated":73756672,"data_stored":73756672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53465403392,"internally_reserved":0,"allocated":87470080,"data_stored":87470080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53469618176,"internally_reserved":0,"allocated":83255296,"data_stored":83255296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53479043072,"internally_reserved":0,"allocated":73830400,"data_stored":73830400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53465395200,"internally_reserved":0,"allocated":87478272,"data_stored":87478272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53493207040,"internally_reserved":0,"allocated":59666432,"data_stored":59666432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:36:35.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 46 16 6253 6237 1.05918 0.945312 0.121137 0.117827 2024-09-22T03:36:35.908 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:36.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:36.063+0000 7f36b3397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:36.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:36.069+0000 7f36b3397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:36.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 47 16 6371 6355 1.05625 0.921875 0.0787198 0.118097 2024-09-22T03:36:37.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 48 16 6514 6498 1.05752 1.11719 0.0603153 0.11805 2024-09-22T03:36:38.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 49 16 6638 6622 1.05571 0.96875 0.144963 0.118233 2024-09-22T03:36:38.248 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:38.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:38.356+0000 7fcee8c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:38.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:38.361+0000 7fcee8c2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:39.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 50 16 6773 6757 1.05568 1.05469 0.120538 0.118271 2024-09-22T03:36:40.113 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:40.114 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:40.114 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:36:40.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 51 16 6912 6896 1.05627 1.08594 0.0566709 0.118167 2024-09-22T03:36:40.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:40.227+0000 7fd3cdcba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:40.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:40.235+0000 7fd3cdcba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:40.465 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:36:40.465 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:36:40.489 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-22T03:36:40.087079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53625395,"num_objects":814,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":6613,"num_write_kb":53688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6563,"ondisk_log_size":6563,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":326420,"kb_used_data":326420,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208864492,"statfs":{"total":214211493888,"available":213877239808,"internally_reserved":0,"allocated":334254080,"data_stored":334254080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5464064,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":667,"num_write_kb":5336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.256330"},"pg_stats":[{"pgid":"10.7","version":"99'400","reported_seq":427,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.855162+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:36:35.855162+0000","last_peered":"2024-09-22T03:36:35.855162+0000","last_clean":"2024-09-22T03:36:35.855162+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:36:35.855162+0000","last_undegraded":"2024-09-22T03:36:35.855162+0000","last_fullsized":"2024-09-22T03:36:35.855162+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"101'447","reported_seq":475,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.699676+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:36:36.699676+0000","last_peered":"2024-09-22T03:36:36.699676+0000","last_clean":"2024-09-22T03:36:36.699676+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:36:36.699676+0000","last_undegraded":"2024-09-22T03:36:36.699676+0000","last_fullsized":"2024-09-22T03:36:36.699676+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"101'447","reported_seq":475,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.028496+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:36:36.028496+0000","last_peered":"2024-09-22T03:36:36.028496+0000","last_clean":"2024-09-22T03:36:36.028496+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:36:36.028496+0000","last_undegraded":"2024-09-22T03:36:36.028496+0000","last_fullsized":"2024-09-22T03:36:36.028496+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"101'391","reported_seq":419,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.769612+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:36:36.769612+0000","last_peered":"2024-09-22T03:36:36.769612+0000","last_clean":"2024-09-22T03:36:36.769612+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:36:36.769612+0000","last_undegraded":"2024-09-22T03:36:36.769612+0000","last_fullsized":"2024-09-22T03:36:36.769612+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"101'383","reported_seq":411,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.994708+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:36:36.994708+0000","last_peered":"2024-09-22T03:36:36.994708+0000","last_clean":"2024-09-22T03:36:36.994708+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:36:36.994708+0000","last_undegraded":"2024-09-22T03:36:36.994708+0000","last_fullsized":"2024-09-22T03:36:36.994708+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"101'375","reported_seq":403,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.399119+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:36:36.399119+0000","last_peered":"2024-09-22T03:36:36.399119+0000","last_clean":"2024-09-22T03:36:36.399119+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:36:36.399119+0000","last_undegraded":"2024-09-22T03:36:36.399119+0000","last_fullsized":"2024-09-22T03:36:36.399119+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"101'375","reported_seq":403,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.491896+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:36:36.491896+0000","last_peered":"2024-09-22T03:36:36.491896+0000","last_clean":"2024-09-22T03:36:36.491896+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:36:36.491896+0000","last_undegraded":"2024-09-22T03:36:36.491896+0000","last_fullsized":"2024-09-22T03:36:36.491896+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"100'400","reported_seq":427,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.808400+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:36:35.808400+0000","last_peered":"2024-09-22T03:36:35.808400+0000","last_clean":"2024-09-22T03:36:35.808400+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:36:35.808400+0000","last_undegraded":"2024-09-22T03:36:35.808400+0000","last_fullsized":"2024-09-22T03:36:35.808400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"101'375","reported_seq":403,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.829434+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:36:36.829434+0000","last_peered":"2024-09-22T03:36:36.829434+0000","last_clean":"2024-09-22T03:36:36.829434+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:36:36.829434+0000","last_undegraded":"2024-09-22T03:36:36.829434+0000","last_fullsized":"2024-09-22T03:36:36.829434+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":3008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":406,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.798865+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:36:35.798865+0000","last_peered":"2024-09-22T03:36:35.798865+0000","last_clean":"2024-09-22T03:36:35.798865+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:36:35.798865+0000","last_undegraded":"2024-09-22T03:36:35.798865+0000","last_fullsized":"2024-09-22T03:36:35.798865+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":272,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.809416+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:36:35.809416+0000","last_peered":"2024-09-22T03:36:35.809416+0000","last_clean":"2024-09-22T03:36:35.809416+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:36:35.809416+0000","last_undegraded":"2024-09-22T03:36:35.809416+0000","last_fullsized":"2024-09-22T03:36:35.809416+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"99'448","reported_seq":475,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.832043+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:36:35.832043+0000","last_peered":"2024-09-22T03:36:35.832043+0000","last_clean":"2024-09-22T03:36:35.832043+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:36:35.832043+0000","last_undegraded":"2024-09-22T03:36:35.832043+0000","last_fullsized":"2024-09-22T03:36:35.832043+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":448,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":271,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.859508+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:36:35.859508+0000","last_peered":"2024-09-22T03:36:35.859508+0000","last_clean":"2024-09-22T03:36:35.859508+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:36:35.859508+0000","last_undegraded":"2024-09-22T03:36:35.859508+0000","last_fullsized":"2024-09-22T03:36:35.859508+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"100'480","reported_seq":507,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.800905+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:36:35.800905+0000","last_peered":"2024-09-22T03:36:35.800905+0000","last_clean":"2024-09-22T03:36:35.800905+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:36:35.800905+0000","last_undegraded":"2024-09-22T03:36:35.800905+0000","last_fullsized":"2024-09-22T03:36:35.800905+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":271,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.807203+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:35.807203+0000","last_peered":"2024-09-22T03:36:35.807203+0000","last_clean":"2024-09-22T03:36:35.807203+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:36:35.807203+0000","last_undegraded":"2024-09-22T03:36:35.807203+0000","last_fullsized":"2024-09-22T03:36:35.807203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"101'399","reported_seq":427,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.977722+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:36:36.977722+0000","last_peered":"2024-09-22T03:36:36.977722+0000","last_clean":"2024-09-22T03:36:36.977722+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:36:36.977722+0000","last_undegraded":"2024-09-22T03:36:36.977722+0000","last_fullsized":"2024-09-22T03:36:36.977722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":273,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.856302+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:35.856302+0000","last_peered":"2024-09-22T03:36:35.856302+0000","last_clean":"2024-09-22T03:36:35.856302+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:36:35.856302+0000","last_undegraded":"2024-09-22T03:36:35.856302+0000","last_fullsized":"2024-09-22T03:36:35.856302+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"101'495","reported_seq":523,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.861350+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:36:36.861350+0000","last_peered":"2024-09-22T03:36:36.861350+0000","last_clean":"2024-09-22T03:36:36.861350+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:36:36.861350+0000","last_undegraded":"2024-09-22T03:36:36.861350+0000","last_fullsized":"2024-09-22T03:36:36.861350+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":271,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.830873+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:35.830873+0000","last_peered":"2024-09-22T03:36:35.830873+0000","last_clean":"2024-09-22T03:36:35.830873+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:36:35.830873+0000","last_undegraded":"2024-09-22T03:36:35.830873+0000","last_fullsized":"2024-09-22T03:36:35.830873+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"101'351","reported_seq":379,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.198231+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:36:36.198231+0000","last_peered":"2024-09-22T03:36:36.198231+0000","last_clean":"2024-09-22T03:36:36.198231+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:36:36.198231+0000","last_undegraded":"2024-09-22T03:36:36.198231+0000","last_fullsized":"2024-09-22T03:36:36.198231+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":352,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":271,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.801274+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:35.801274+0000","last_peered":"2024-09-22T03:36:35.801274+0000","last_clean":"2024-09-22T03:36:35.801274+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:36:35.801274+0000","last_undegraded":"2024-09-22T03:36:35.801274+0000","last_fullsized":"2024-09-22T03:36:35.801274+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"101'361","reported_seq":389,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:36.946237+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:36:36.946237+0000","last_peered":"2024-09-22T03:36:36.946237+0000","last_clean":"2024-09-22T03:36:36.946237+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:36:36.946237+0000","last_undegraded":"2024-09-22T03:36:36.946237+0000","last_fullsized":"2024-09-22T03:36:36.946237+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2965504,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":362,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":271,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.832619+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:36:35.832619+0000","last_peered":"2024-09-22T03:36:35.832619+0000","last_clean":"2024-09-22T03:36:35.832619+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:36:35.832619+0000","last_undegraded":"2024-09-22T03:36:35.832619+0000","last_fullsized":"2024-09-22T03:36:35.832619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"100'336","reported_seq":363,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.811627+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:36:35.811627+0000","last_peered":"2024-09-22T03:36:35.811627+0000","last_clean":"2024-09-22T03:36:35.811627+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:36:35.811627+0000","last_undegraded":"2024-09-22T03:36:35.811627+0000","last_fullsized":"2024-09-22T03:36:35.811627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":271,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-22T03:36:35.809139+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:36:35.809139+0000","last_peered":"2024-09-22T03:36:35.809139+0000","last_clean":"2024-09-22T03:36:35.809139+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:36:35.809139+0000","last_undegraded":"2024-09-22T03:36:35.809139+0000","last_fullsized":"2024-09-22T03:36:35.809139+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":53035008,"num_objects":810,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":51792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213872476160,"internally_reserved":0,"allocated":339017728,"data_stored":339017728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6463,"ondisk_log_size":6463,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213872476160,"internally_reserved":0,"allocated":339017728,"data_stored":339017728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53485928448,"internally_reserved":0,"allocated":66945024,"data_stored":66945024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":64364,"kb_used_data":64364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52233364,"statfs":{"total":53552873472,"available":53486964736,"internally_reserved":0,"allocated":65908736,"data_stored":65908736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":94328,"kb_used_data":94328,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52203400,"statfs":{"total":53552873472,"available":53456281600,"internally_reserved":0,"allocated":96591872,"data_stored":96591872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":79332,"kb_used_data":79332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52218396,"statfs":{"total":53552873472,"available":53471637504,"internally_reserved":0,"allocated":81235968,"data_stored":81235968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":88396,"kb_used_data":88396,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52209332,"statfs":{"total":53552873472,"available":53462355968,"internally_reserved":0,"allocated":90517504,"data_stored":90517504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53485928448,"internally_reserved":0,"allocated":66945024,"data_stored":66945024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53461131264,"internally_reserved":0,"allocated":91742208,"data_stored":91742208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53470539776,"internally_reserved":0,"allocated":82333696,"data_stored":82333696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53454876672,"internally_reserved":0,"allocated":97996800,"data_stored":97996800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53485928448,"internally_reserved":0,"allocated":66945024,"data_stored":66945024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53461131264,"internally_reserved":0,"allocated":91742208,"data_stored":91742208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53470539776,"internally_reserved":0,"allocated":82333696,"data_stored":82333696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53454876672,"internally_reserved":0,"allocated":97996800,"data_stored":97996800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53485928448,"internally_reserved":0,"allocated":66945024,"data_stored":66945024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:36:40.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:40.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:40.686+0000 7f0925863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:40.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:40.692+0000 7f0925863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:41.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 52 16 7057 7041 1.05774 1.13281 0.13171 0.118016 2024-09-22T03:36:42.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 53 16 7184 7168 1.05651 0.992188 0.153501 0.118154 2024-09-22T03:36:42.861 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:42.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:42.976+0000 7f9a7744f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:42.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:42.982+0000 7f9a7744f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:43.231 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 54 16 7293 7277 1.05271 0.851562 0.0395437 0.118496 2024-09-22T03:36:44.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 55 16 7438 7422 1.05416 1.13281 0.17495 0.11837 2024-09-22T03:36:45.230 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:45.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 56 16 7583 7567 1.05557 1.13281 0.102612 0.118278 2024-09-22T03:36:45.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:45.355+0000 7f3ab9f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:45.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:45.362+0000 7f3ab9f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:45.491 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:45.491 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:45.493 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:36:45.493 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:36:45.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:45.617+0000 7fde1e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:45.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:45.626+0000 7fde1e597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:45.746 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:45.747 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:45.747 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:45.747 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:45.747 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:45.747 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:45.750 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:45.750 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:45.750 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:36:45.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:45.872+0000 7f9a7f663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:45.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:45.883+0000 7f9a7f663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:45.933 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:45.933 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:45.933 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:45.933 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:45.933 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:45.934 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:45.934 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:45.934 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:45.934 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:45.934 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:45.934 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:45.934 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:45.936 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:45.936 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:45.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:36:46.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:46.082+0000 7f1714757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:46.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:46.092+0000 7f1714757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:46.177 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:46.177 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:46.177 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:46.177 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:46.178 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:46.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:46.180 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:46.180 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:36:46.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 57 16 7713 7697 1.05486 1.01562 0.142669 0.118323 2024-09-22T03:36:46.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:46.306+0000 7f587a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:46.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:46.316+0000 7f587a997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:46.341 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:46.341 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:46.341 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:46.341 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:46.342 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:46.344 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:46.345 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:47.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 58 16 7851 7835 1.05526 1.07812 0.137024 0.118291 2024-09-22T03:36:47.680 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:47.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:47.812+0000 7f11dbf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:47.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:47.819+0000 7f11dbf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:48.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 59 16 7998 7982 1.05684 1.14844 0.165834 0.118152 2024-09-22T03:36:49.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:36:49.232142+0000 min lat: 0.0145288 max lat: 0.54404 avg lat: 0.118254 2024-09-22T03:36:49.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:36:49.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 60 16 8123 8107 1.0555 0.976562 0.120249 0.118254 2024-09-22T03:36:50.051 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:50.161+0000 7f1b2f64f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:50.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:50.167+0000 7f1b2f64f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:50.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 61 16 8270 8254 1.05702 1.14844 0.0502936 0.118042 2024-09-22T03:36:51.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 62 16 8424 8408 1.05938 1.20312 0.0791555 0.117832 2024-09-22T03:36:51.345 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:51.345 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:51.345 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:36:51.346 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:36:51.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:51.478+0000 7fc79810e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:51.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:51.491+0000 7fc79810e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:51.554 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:51.554 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:51.554 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:51.555 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:51.558 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:51.559 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:51.559 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:36:51.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:51.678+0000 7f3051997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:51.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:51.687+0000 7f3051997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:51.772 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:51.773 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:51.776 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:51.776 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:51.776 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:36:51.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:51.921+0000 7f935d397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:51.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:51.931+0000 7f935d397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:51.974 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:51.975 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:51.975 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:51.975 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:51.977 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:51.977 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:51.977 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:36:52.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:52.113+0000 7fdfb50d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:52.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:52.124+0000 7fdfb50d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:36:52.176 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:36:52.178 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:36:52.179 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:36:52.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 63 16 8560 8544 1.05943 1.0625 0.13486 0.117803 2024-09-22T03:36:52.371 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:52.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:52.479+0000 7f9afd6ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:52.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:52.484+0000 7f9afd6ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:53.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 64 16 8687 8671 1.05837 0.992188 0.0721926 0.117995 2024-09-22T03:36:54.232 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 65 16 8824 8808 1.05856 1.07031 0.086324 0.117964 2024-09-22T03:36:54.578 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:54.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:54.728+0000 7fd496d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:54.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:54.735+0000 7fd496d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:55.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 66 16 8956 8940 1.05814 1.03125 0.0419413 0.117962 2024-09-22T03:36:56.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 67 16 9082 9066 1.05704 0.984375 0.0858215 0.118096 2024-09-22T03:36:57.050 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:36:57.179 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:36:57.180 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:36:57.180 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:36:57.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:57.177+0000 7fd80f26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:57.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:57.183+0000 7fd80f26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:57.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 68 16 9231 9215 1.05861 1.16406 0.142213 0.117972 2024-09-22T03:36:57.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:57.341+0000 7f83180ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:57.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:57.352+0000 7f83180ba640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:57.603 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:36:57.604 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:36:57.617 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-22T03:36:56.090310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":70427187,"num_objects":1071,"num_object_clones":0,"num_object_copies":3213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":8664,"num_write_kb":70096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8612,"ondisk_log_size":8612,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":428536,"kb_used_data":428536,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208762376,"statfs":{"total":214211493888,"available":213772673024,"internally_reserved":0,"allocated":438820864,"data_stored":438820864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5750784,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":702,"num_write_kb":5616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001631"},"pg_stats":[{"pgid":"10.7","version":"104'495","reported_seq":532,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.661514+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:36:51.661514+0000","last_peered":"2024-09-22T03:36:51.661514+0000","last_clean":"2024-09-22T03:36:51.661514+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:36:51.661514+0000","last_undegraded":"2024-09-22T03:36:51.661514+0000","last_fullsized":"2024-09-22T03:36:51.661514+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"103'576","reported_seq":612,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.814673+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:36:50.814673+0000","last_peered":"2024-09-22T03:36:50.814673+0000","last_clean":"2024-09-22T03:36:50.814673+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:36:50.814673+0000","last_undegraded":"2024-09-22T03:36:50.814673+0000","last_fullsized":"2024-09-22T03:36:50.814673+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"103'616","reported_seq":652,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.835321+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:36:50.835321+0000","last_peered":"2024-09-22T03:36:50.835321+0000","last_clean":"2024-09-22T03:36:50.835321+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:36:50.835321+0000","last_undegraded":"2024-09-22T03:36:50.835321+0000","last_fullsized":"2024-09-22T03:36:50.835321+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5046272,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"104'567","reported_seq":604,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.528482+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:36:51.528482+0000","last_peered":"2024-09-22T03:36:51.528482+0000","last_clean":"2024-09-22T03:36:51.528482+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:36:51.528482+0000","last_undegraded":"2024-09-22T03:36:51.528482+0000","last_fullsized":"2024-09-22T03:36:51.528482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":568,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"104'495","reported_seq":532,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.160416+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:36:51.160416+0000","last_peered":"2024-09-22T03:36:51.160416+0000","last_clean":"2024-09-22T03:36:51.160416+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:36:51.160416+0000","last_undegraded":"2024-09-22T03:36:51.160416+0000","last_fullsized":"2024-09-22T03:36:51.160416+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"103'440","reported_seq":476,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.840680+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:36:50.840680+0000","last_peered":"2024-09-22T03:36:50.840680+0000","last_clean":"2024-09-22T03:36:50.840680+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:36:50.840680+0000","last_undegraded":"2024-09-22T03:36:50.840680+0000","last_fullsized":"2024-09-22T03:36:50.840680+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"104'528","reported_seq":565,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.849795+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:36:51.849795+0000","last_peered":"2024-09-22T03:36:51.849795+0000","last_clean":"2024-09-22T03:36:51.849795+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:36:51.849795+0000","last_undegraded":"2024-09-22T03:36:51.849795+0000","last_fullsized":"2024-09-22T03:36:51.849795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4333568,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":529,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"104'527","reported_seq":564,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.759665+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:36:51.759665+0000","last_peered":"2024-09-22T03:36:51.759665+0000","last_clean":"2024-09-22T03:36:51.759665+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:36:51.759665+0000","last_undegraded":"2024-09-22T03:36:51.759665+0000","last_fullsized":"2024-09-22T03:36:51.759665+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"104'531","reported_seq":568,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:52.006599+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:36:52.006599+0000","last_peered":"2024-09-22T03:36:52.006599+0000","last_clean":"2024-09-22T03:36:52.006599+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:36:52.006599+0000","last_undegraded":"2024-09-22T03:36:52.006599+0000","last_fullsized":"2024-09-22T03:36:52.006599+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4358144,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":532,"num_write_kb":4256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":415,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.838099+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:36:50.838099+0000","last_peered":"2024-09-22T03:36:50.838099+0000","last_clean":"2024-09-22T03:36:50.838099+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:36:50.838099+0000","last_undegraded":"2024-09-22T03:36:50.838099+0000","last_fullsized":"2024-09-22T03:36:50.838099+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":281,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.815596+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:36:50.815596+0000","last_peered":"2024-09-22T03:36:50.815596+0000","last_clean":"2024-09-22T03:36:50.815596+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:36:50.815596+0000","last_undegraded":"2024-09-22T03:36:50.815596+0000","last_fullsized":"2024-09-22T03:36:50.815596+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"104'575","reported_seq":612,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.866752+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:36:50.866752+0000","last_peered":"2024-09-22T03:36:50.866752+0000","last_clean":"2024-09-22T03:36:50.866752+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:36:50.866752+0000","last_undegraded":"2024-09-22T03:36:50.866752+0000","last_fullsized":"2024-09-22T03:36:50.866752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":576,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":280,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.833060+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:36:50.833060+0000","last_peered":"2024-09-22T03:36:50.833060+0000","last_clean":"2024-09-22T03:36:50.833060+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:36:50.833060+0000","last_undegraded":"2024-09-22T03:36:50.833060+0000","last_fullsized":"2024-09-22T03:36:50.833060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"104'591","reported_seq":628,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.221789+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:36:51.221789+0000","last_peered":"2024-09-22T03:36:51.221789+0000","last_clean":"2024-09-22T03:36:51.221789+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:36:51.221789+0000","last_undegraded":"2024-09-22T03:36:51.221789+0000","last_fullsized":"2024-09-22T03:36:51.221789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.811939+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:50.811939+0000","last_peered":"2024-09-22T03:36:50.811939+0000","last_clean":"2024-09-22T03:36:50.811939+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:36:50.811939+0000","last_undegraded":"2024-09-22T03:36:50.811939+0000","last_fullsized":"2024-09-22T03:36:50.811939+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"104'527","reported_seq":564,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.584566+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:36:51.584566+0000","last_peered":"2024-09-22T03:36:51.584566+0000","last_clean":"2024-09-22T03:36:51.584566+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:36:51.584566+0000","last_undegraded":"2024-09-22T03:36:51.584566+0000","last_fullsized":"2024-09-22T03:36:51.584566+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":528,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":282,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.830224+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:36:50.830224+0000","last_peered":"2024-09-22T03:36:50.830224+0000","last_clean":"2024-09-22T03:36:50.830224+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:36:50.830224+0000","last_undegraded":"2024-09-22T03:36:50.830224+0000","last_fullsized":"2024-09-22T03:36:50.830224+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"104'631","reported_seq":668,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.900816+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:36:51.900816+0000","last_peered":"2024-09-22T03:36:51.900816+0000","last_clean":"2024-09-22T03:36:51.900816+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:36:51.900816+0000","last_undegraded":"2024-09-22T03:36:51.900816+0000","last_fullsized":"2024-09-22T03:36:51.900816+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":280,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.841119+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:50.841119+0000","last_peered":"2024-09-22T03:36:50.841119+0000","last_clean":"2024-09-22T03:36:50.841119+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:36:50.841119+0000","last_undegraded":"2024-09-22T03:36:50.841119+0000","last_fullsized":"2024-09-22T03:36:50.841119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"104'511","reported_seq":548,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.007065+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:36:51.007065+0000","last_peered":"2024-09-22T03:36:51.007065+0000","last_clean":"2024-09-22T03:36:51.007065+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:36:51.007065+0000","last_undegraded":"2024-09-22T03:36:51.007065+0000","last_fullsized":"2024-09-22T03:36:51.007065+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":280,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.841744+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:36:50.841744+0000","last_peered":"2024-09-22T03:36:50.841744+0000","last_clean":"2024-09-22T03:36:50.841744+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:36:50.841744+0000","last_undegraded":"2024-09-22T03:36:50.841744+0000","last_fullsized":"2024-09-22T03:36:50.841744+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"104'487","reported_seq":524,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.532255+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:36:51.532255+0000","last_peered":"2024-09-22T03:36:51.532255+0000","last_clean":"2024-09-22T03:36:51.532255+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:36:51.532255+0000","last_undegraded":"2024-09-22T03:36:51.532255+0000","last_fullsized":"2024-09-22T03:36:51.532255+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":488,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":280,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.838557+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:36:50.838557+0000","last_peered":"2024-09-22T03:36:50.838557+0000","last_clean":"2024-09-22T03:36:50.838557+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:36:50.838557+0000","last_undegraded":"2024-09-22T03:36:50.838557+0000","last_fullsized":"2024-09-22T03:36:50.838557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"104'415","reported_seq":452,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:51.754959+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:36:51.754959+0000","last_peered":"2024-09-22T03:36:51.754959+0000","last_clean":"2024-09-22T03:36:51.754959+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:36:51.754959+0000","last_undegraded":"2024-09-22T03:36:51.754959+0000","last_fullsized":"2024-09-22T03:36:51.754959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T03:36:50.814426+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:36:50.814426+0000","last_peered":"2024-09-22T03:36:50.814426+0000","last_clean":"2024-09-22T03:36:50.814426+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:36:50.814426+0000","last_undegraded":"2024-09-22T03:36:50.814426+0000","last_fullsized":"2024-09-22T03:36:50.814426+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":69836800,"num_objects":1067,"num_object_clones":0,"num_object_copies":3201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8525,"num_write_kb":68200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213768077312,"internally_reserved":0,"allocated":443416576,"data_stored":443416576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8512,"ondisk_log_size":8512,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213768077312,"internally_reserved":0,"allocated":443416576,"data_stored":443416576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53466841088,"internally_reserved":0,"allocated":86032384,"data_stored":86032384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":83364,"kb_used_data":83364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52214364,"statfs":{"total":53552873472,"available":53467508736,"internally_reserved":0,"allocated":85364736,"data_stored":85364736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":125528,"kb_used_data":125528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52172200,"statfs":{"total":53552873472,"available":53424332800,"internally_reserved":0,"allocated":128540672,"data_stored":128540672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":105260,"kb_used_data":105260,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52192468,"statfs":{"total":53552873472,"available":53445087232,"internally_reserved":0,"allocated":107786240,"data_stored":107786240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":114384,"kb_used_data":114384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52183344,"statfs":{"total":53552873472,"available":53435744256,"internally_reserved":0,"allocated":117129216,"data_stored":117129216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53466841088,"internally_reserved":0,"allocated":86032384,"data_stored":86032384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53434667008,"internally_reserved":0,"allocated":118206464,"data_stored":118206464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53443747840,"internally_reserved":0,"allocated":109125632,"data_stored":109125632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53422821376,"internally_reserved":0,"allocated":130052096,"data_stored":130052096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53466841088,"internally_reserved":0,"allocated":86032384,"data_stored":86032384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53434667008,"internally_reserved":0,"allocated":118206464,"data_stored":118206464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53443747840,"internally_reserved":0,"allocated":109125632,"data_stored":109125632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53422821376,"internally_reserved":0,"allocated":130052096,"data_stored":130052096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53466841088,"internally_reserved":0,"allocated":86032384,"data_stored":86032384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:36:58.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 69 16 9365 9349 1.05844 1.04688 0.158955 0.117972 2024-09-22T03:36:59.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 70 16 9479 9463 1.05604 0.890625 0.165704 0.118248 2024-09-22T03:36:59.430 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:36:59.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:59.557+0000 7f8ae6d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:36:59.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:36:59.563+0000 7f8ae6d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:00.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 71 16 9626 9610 1.05734 1.14844 0.109181 0.11811 2024-09-22T03:37:01.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 72 16 9734 9718 1.05437 0.84375 0.033628 0.118379 2024-09-22T03:37:01.834 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:37:01.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:01.964+0000 7fc97c997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:01.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:01.971+0000 7fc97c997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:02.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 73 16 9861 9845 1.05352 0.992188 0.171169 0.118492 2024-09-22T03:37:02.619 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:37:02.620 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:37:02.620 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:37:02.620 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:37:02.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:02.771+0000 7fdf5984f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:02.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:02.783+0000 7fdf5984f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:02.862 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:02.862 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:02.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:37:02.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:02.980+0000 7fa8df517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:02.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:02.990+0000 7fa8df517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:03.036 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:03.036 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:03.036 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:03.036 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:03.036 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:03.036 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:03.037 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:03.037 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:03.037 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:03.037 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:03.037 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:03.037 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:03.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:03.041 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:03.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:37:03.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:03.168+0000 7f762b797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:03.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:03.178+0000 7f762b797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:03.205 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:03.206 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:03.206 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:03.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:03.209 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:03.209 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:37:03.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 74 16 9986 9970 1.05248 0.976562 0.0414038 0.118662 2024-09-22T03:37:03.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:03.321+0000 7ff28fb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:03.330+0000 7ff28fb57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:03.386 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:03.387 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:03.389 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:03.390 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:04.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:37:04.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 75 16 10116 10100 1.05199 1.01562 0.133125 0.118693 2024-09-22T03:37:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:04.315+0000 7f485f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:04.320+0000 7f485f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:05.233 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 76 16 10241 10225 1.05099 0.976562 0.142641 0.118834 2024-09-22T03:37:06.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 77 16 10383 10367 1.05175 1.10938 0.182308 0.118699 2024-09-22T03:37:06.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:37:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:06.665+0000 7f9590997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:06.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:06.674+0000 7f9590997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:07.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 78 16 10516 10500 1.05159 1.03906 0.149713 0.118713 2024-09-22T03:37:08.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 79 16 10640 10624 1.05054 0.96875 0.156619 0.118871 2024-09-22T03:37:08.390 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:37:08.390 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:37:08.390 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:37:08.390 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:37:08.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:08.504+0000 7fdd8dcd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:08.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:08.514+0000 7fdd8dcd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:08.538 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:08.538 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:08.538 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:08.539 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:08.542 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:08.542 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:08.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:37:08.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:08.666+0000 7f45342b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:08.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:08.682+0000 7f45342b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:08.786 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:08.787 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:08.787 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:08.787 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:08.787 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:08.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:08.789 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:08.790 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:37:08.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:08.946+0000 7fc4486fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:08.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:08.955+0000 7fc4486fa640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:08.972 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:08.972 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:08.972 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:08.972 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:08.972 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:08.973 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:08.973 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:08.973 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:08.973 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:08.973 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:08.973 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:08.973 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:08.975 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:08.976 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:08.976 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:37:08.997 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:37:09.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:09.125+0000 7fdf5f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:09.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:09.140+0000 7fdf5f957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:09.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:09.139+0000 7fda51e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:09.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:09.146+0000 7fda51e2c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:09.159 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:09.159 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:09.160 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:09.166 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:09.167 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:09.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:2024-09-22T03:37:09.234005+0000 min lat: 0.0145288 max lat: 0.54404 avg lat: 0.118915 2024-09-22T03:37:09.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-22T03:37:09.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 80 16 10768 10752 1.0499 1 0.039707 0.118915 2024-09-22T03:37:10.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 81 16 10889 10873 1.04861 0.945312 0.181704 0.119047 2024-09-22T03:37:11.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 82 16 11031 11015 1.04935 1.10938 0.1551 0.118998 2024-09-22T03:37:11.402 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:37:11.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:11.543+0000 7f226be23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:11.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:11.549+0000 7f226be23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:12.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 83 16 11167 11151 1.04951 1.0625 0.0761518 0.118988 2024-09-22T03:37:13.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 84 16 11299 11283 1.04929 1.03125 0.134182 0.119028 2024-09-22T03:37:13.682 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:37:13.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:13.793+0000 7f7133797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:13.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:13.798+0000 7f7133797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:14.168 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:37:14.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:37:14.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 85 16 11438 11422 1.04972 1.08594 0.0850295 0.11901 2024-09-22T03:37:14.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:14.334+0000 7fefdcf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:14.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:14.344+0000 7fefdcf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:14.786 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:37:14.809 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:37:14.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:14.920+0000 7f1579a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:14.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:14.931+0000 7f1579a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:15.234 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 86 16 11568 11552 1.04932 1.01562 0.0742823 0.119002 2024-09-22T03:37:15.785 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:37:15.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:37:15.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:15.942+0000 7f5761e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:15.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:15.954+0000 7f5761e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:15.961 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:37:16.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:16.076+0000 7fc6b2d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:16.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:16.081+0000 7fc6b2d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:16.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 87 16 11704 11688 1.04947 1.0625 0.170047 0.119001 2024-09-22T03:37:16.800 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:37:16.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:37:16.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:16.961+0000 7fac8fcb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:16.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:16.977+0000 7fac8fcb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:17.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 88 16 11808 11792 1.04678 0.8125 0.196352 0.119277 2024-09-22T03:37:17.801 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:37:17.826 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:37:17.826 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:17.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:17.941+0000 7fd385b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:17.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:17.949+0000 7fd385b0e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:18.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 89 16 11937 11921 1.04634 1.00781 0.0709086 0.119365 2024-09-22T03:37:18.274 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:18.274 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:18.293 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-09-22T03:37:18.094293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97133107,"num_objects":1478,"num_object_clones":0,"num_object_copies":4434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11924,"num_write_kb":96176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11881,"ondisk_log_size":11881,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":591828,"kb_used_data":591828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208599084,"statfs":{"total":214211493888,"available":213605462016,"internally_reserved":0,"allocated":606031872,"data_stored":606031872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5332992,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":651,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000715"},"pg_stats":[{"pgid":"10.7","version":"107'704","reported_seq":752,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.852439+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:37:16.852439+0000","last_peered":"2024-09-22T03:37:16.852439+0000","last_clean":"2024-09-22T03:37:16.852439+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:37:16.852439+0000","last_undegraded":"2024-09-22T03:37:16.852439+0000","last_fullsized":"2024-09-22T03:37:16.852439+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"107'768","reported_seq":816,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.855964+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:37:16.855964+0000","last_peered":"2024-09-22T03:37:16.855964+0000","last_clean":"2024-09-22T03:37:16.855964+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:37:16.855964+0000","last_undegraded":"2024-09-22T03:37:16.855964+0000","last_fullsized":"2024-09-22T03:37:16.855964+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"107'896","reported_seq":944,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.890944+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:37:16.890944+0000","last_peered":"2024-09-22T03:37:16.890944+0000","last_clean":"2024-09-22T03:37:16.890944+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:37:16.890944+0000","last_undegraded":"2024-09-22T03:37:16.890944+0000","last_fullsized":"2024-09-22T03:37:16.890944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"107'808","reported_seq":856,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.816817+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:37:16.816817+0000","last_peered":"2024-09-22T03:37:16.816817+0000","last_clean":"2024-09-22T03:37:16.816817+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:37:16.816817+0000","last_undegraded":"2024-09-22T03:37:16.816817+0000","last_fullsized":"2024-09-22T03:37:16.816817+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"107'656","reported_seq":704,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.857367+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:37:16.857367+0000","last_peered":"2024-09-22T03:37:16.857367+0000","last_clean":"2024-09-22T03:37:16.857367+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:37:16.857367+0000","last_undegraded":"2024-09-22T03:37:16.857367+0000","last_fullsized":"2024-09-22T03:37:16.857367+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"107'704","reported_seq":752,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.818434+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:37:16.818434+0000","last_peered":"2024-09-22T03:37:16.818434+0000","last_clean":"2024-09-22T03:37:16.818434+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:37:16.818434+0000","last_undegraded":"2024-09-22T03:37:16.818434+0000","last_fullsized":"2024-09-22T03:37:16.818434+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"106'680","reported_seq":728,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.856353+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:37:16.856353+0000","last_peered":"2024-09-22T03:37:16.856353+0000","last_clean":"2024-09-22T03:37:16.856353+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:37:16.856353+0000","last_undegraded":"2024-09-22T03:37:16.856353+0000","last_fullsized":"2024-09-22T03:37:16.856353+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"108'767","reported_seq":816,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.965257+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:37:16.965257+0000","last_peered":"2024-09-22T03:37:16.965257+0000","last_clean":"2024-09-22T03:37:16.965257+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:37:16.965257+0000","last_undegraded":"2024-09-22T03:37:16.965257+0000","last_fullsized":"2024-09-22T03:37:16.965257+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"106'720","reported_seq":769,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.858438+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:37:16.858438+0000","last_peered":"2024-09-22T03:37:16.858438+0000","last_clean":"2024-09-22T03:37:16.858438+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:37:16.858438+0000","last_undegraded":"2024-09-22T03:37:16.858438+0000","last_fullsized":"2024-09-22T03:37:16.858438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":427,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.817067+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:16.817067+0000","last_peered":"2024-09-22T03:37:16.817067+0000","last_clean":"2024-09-22T03:37:16.817067+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:16.817067+0000","last_undegraded":"2024-09-22T03:37:16.817067+0000","last_fullsized":"2024-09-22T03:37:16.817067+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":293,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.856727+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:16.856727+0000","last_peered":"2024-09-22T03:37:16.856727+0000","last_clean":"2024-09-22T03:37:16.856727+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:16.856727+0000","last_undegraded":"2024-09-22T03:37:16.856727+0000","last_fullsized":"2024-09-22T03:37:16.856727+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"108'791","reported_seq":840,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.981897+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:37:16.981897+0000","last_peered":"2024-09-22T03:37:16.981897+0000","last_clean":"2024-09-22T03:37:16.981897+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:37:16.981897+0000","last_undegraded":"2024-09-22T03:37:16.981897+0000","last_fullsized":"2024-09-22T03:37:16.981897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.860371+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:16.860371+0000","last_peered":"2024-09-22T03:37:16.860371+0000","last_clean":"2024-09-22T03:37:16.860371+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:16.860371+0000","last_undegraded":"2024-09-22T03:37:16.860371+0000","last_fullsized":"2024-09-22T03:37:16.860371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"106'776","reported_seq":824,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.818647+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:37:16.818647+0000","last_peered":"2024-09-22T03:37:16.818647+0000","last_clean":"2024-09-22T03:37:16.818647+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:37:16.818647+0000","last_undegraded":"2024-09-22T03:37:16.818647+0000","last_fullsized":"2024-09-22T03:37:16.818647+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.852841+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:16.852841+0000","last_peered":"2024-09-22T03:37:16.852841+0000","last_clean":"2024-09-22T03:37:16.852841+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:16.852841+0000","last_undegraded":"2024-09-22T03:37:16.852841+0000","last_fullsized":"2024-09-22T03:37:16.852841+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"106'712","reported_seq":760,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.857910+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:37:16.857910+0000","last_peered":"2024-09-22T03:37:16.857910+0000","last_clean":"2024-09-22T03:37:16.857910+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:37:16.857910+0000","last_undegraded":"2024-09-22T03:37:16.857910+0000","last_fullsized":"2024-09-22T03:37:16.857910+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":294,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.852166+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:16.852166+0000","last_peered":"2024-09-22T03:37:16.852166+0000","last_clean":"2024-09-22T03:37:16.852166+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:16.852166+0000","last_undegraded":"2024-09-22T03:37:16.852166+0000","last_fullsized":"2024-09-22T03:37:16.852166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"108'807","reported_seq":856,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.877209+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:37:16.877209+0000","last_peered":"2024-09-22T03:37:16.877209+0000","last_clean":"2024-09-22T03:37:16.877209+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:37:16.877209+0000","last_undegraded":"2024-09-22T03:37:16.877209+0000","last_fullsized":"2024-09-22T03:37:16.877209+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.894088+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:16.894088+0000","last_peered":"2024-09-22T03:37:16.894088+0000","last_clean":"2024-09-22T03:37:16.894088+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:16.894088+0000","last_undegraded":"2024-09-22T03:37:16.894088+0000","last_fullsized":"2024-09-22T03:37:16.894088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"107'720","reported_seq":768,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.856187+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:37:16.856187+0000","last_peered":"2024-09-22T03:37:16.856187+0000","last_clean":"2024-09-22T03:37:16.856187+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:37:16.856187+0000","last_undegraded":"2024-09-22T03:37:16.856187+0000","last_fullsized":"2024-09-22T03:37:16.856187+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.820375+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:16.820375+0000","last_peered":"2024-09-22T03:37:16.820375+0000","last_clean":"2024-09-22T03:37:16.820375+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:16.820375+0000","last_undegraded":"2024-09-22T03:37:16.820375+0000","last_fullsized":"2024-09-22T03:37:16.820375+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"106'680","reported_seq":728,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.862340+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:37:16.862340+0000","last_peered":"2024-09-22T03:37:16.862340+0000","last_clean":"2024-09-22T03:37:16.862340+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:37:16.862340+0000","last_undegraded":"2024-09-22T03:37:16.862340+0000","last_fullsized":"2024-09-22T03:37:16.862340+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.894273+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:16.894273+0000","last_peered":"2024-09-22T03:37:16.894273+0000","last_clean":"2024-09-22T03:37:16.894273+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:16.894273+0000","last_undegraded":"2024-09-22T03:37:16.894273+0000","last_fullsized":"2024-09-22T03:37:16.894273+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"107'592","reported_seq":640,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.855975+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:37:16.855975+0000","last_peered":"2024-09-22T03:37:16.855975+0000","last_clean":"2024-09-22T03:37:16.855975+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:37:16.855975+0000","last_undegraded":"2024-09-22T03:37:16.855975+0000","last_fullsized":"2024-09-22T03:37:16.855975+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.858723+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:16.858723+0000","last_peered":"2024-09-22T03:37:16.858723+0000","last_clean":"2024-09-22T03:37:16.858723+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:16.858723+0000","last_undegraded":"2024-09-22T03:37:16.858723+0000","last_fullsized":"2024-09-22T03:37:16.858723+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":96542720,"num_objects":1474,"num_object_clones":0,"num_object_copies":4422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11785,"num_write_kb":94280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213601562624,"internally_reserved":0,"allocated":609931264,"data_stored":609931264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11781,"ondisk_log_size":11781,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213601570816,"internally_reserved":0,"allocated":609923072,"data_stored":609923072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":115264,"kb_used_data":115264,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52182464,"statfs":{"total":53552873472,"available":53434843136,"internally_reserved":0,"allocated":118030336,"data_stored":118030336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":172576,"kb_used_data":172576,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125152,"statfs":{"total":53552873472,"available":53376155648,"internally_reserved":0,"allocated":176717824,"data_stored":176717824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":147080,"kb_used_data":147080,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52150648,"statfs":{"total":53552873472,"available":53402263552,"internally_reserved":0,"allocated":150609920,"data_stored":150609920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":156908,"kb_used_data":156908,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52140820,"statfs":{"total":53552873472,"available":53392199680,"internally_reserved":0,"allocated":160673792,"data_stored":160673792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53391048704,"internally_reserved":0,"allocated":161824768,"data_stored":161824768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53401284608,"internally_reserved":0,"allocated":151588864,"data_stored":151588864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53375164416,"internally_reserved":0,"allocated":177709056,"data_stored":177709056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53391040512,"internally_reserved":0,"allocated":161832960,"data_stored":161832960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53401284608,"internally_reserved":0,"allocated":151588864,"data_stored":151588864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53375164416,"internally_reserved":0,"allocated":177709056,"data_stored":177709056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:18.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:18.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:37:18.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:18.434+0000 7f844ec6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:18.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:18.450+0000 7f844ec6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:18.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:18.471+0000 7face3423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:18.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:18.478+0000 7face3423640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:18.734 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:18.734 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:18.756 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-09-22T03:37:18.094293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97133107,"num_objects":1478,"num_object_clones":0,"num_object_copies":4434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":11924,"num_write_kb":96176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11881,"ondisk_log_size":11881,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":591828,"kb_used_data":591828,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208599084,"statfs":{"total":214211493888,"available":213605462016,"internally_reserved":0,"allocated":606031872,"data_stored":606031872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5332992,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":651,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000715"},"pg_stats":[{"pgid":"10.7","version":"107'704","reported_seq":752,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.852439+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:37:16.852439+0000","last_peered":"2024-09-22T03:37:16.852439+0000","last_clean":"2024-09-22T03:37:16.852439+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:37:16.852439+0000","last_undegraded":"2024-09-22T03:37:16.852439+0000","last_fullsized":"2024-09-22T03:37:16.852439+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"107'768","reported_seq":816,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.855964+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:37:16.855964+0000","last_peered":"2024-09-22T03:37:16.855964+0000","last_clean":"2024-09-22T03:37:16.855964+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:37:16.855964+0000","last_undegraded":"2024-09-22T03:37:16.855964+0000","last_fullsized":"2024-09-22T03:37:16.855964+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"107'896","reported_seq":944,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.890944+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:37:16.890944+0000","last_peered":"2024-09-22T03:37:16.890944+0000","last_clean":"2024-09-22T03:37:16.890944+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:37:16.890944+0000","last_undegraded":"2024-09-22T03:37:16.890944+0000","last_fullsized":"2024-09-22T03:37:16.890944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":896,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"107'808","reported_seq":856,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.816817+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:37:16.816817+0000","last_peered":"2024-09-22T03:37:16.816817+0000","last_clean":"2024-09-22T03:37:16.816817+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:37:16.816817+0000","last_undegraded":"2024-09-22T03:37:16.816817+0000","last_fullsized":"2024-09-22T03:37:16.816817+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"107'656","reported_seq":704,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.857367+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:37:16.857367+0000","last_peered":"2024-09-22T03:37:16.857367+0000","last_clean":"2024-09-22T03:37:16.857367+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:37:16.857367+0000","last_undegraded":"2024-09-22T03:37:16.857367+0000","last_fullsized":"2024-09-22T03:37:16.857367+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"107'704","reported_seq":752,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.818434+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:37:16.818434+0000","last_peered":"2024-09-22T03:37:16.818434+0000","last_clean":"2024-09-22T03:37:16.818434+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:37:16.818434+0000","last_undegraded":"2024-09-22T03:37:16.818434+0000","last_fullsized":"2024-09-22T03:37:16.818434+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"106'680","reported_seq":728,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.856353+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:37:16.856353+0000","last_peered":"2024-09-22T03:37:16.856353+0000","last_clean":"2024-09-22T03:37:16.856353+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:37:16.856353+0000","last_undegraded":"2024-09-22T03:37:16.856353+0000","last_fullsized":"2024-09-22T03:37:16.856353+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"108'767","reported_seq":816,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.965257+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:37:16.965257+0000","last_peered":"2024-09-22T03:37:16.965257+0000","last_clean":"2024-09-22T03:37:16.965257+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:37:16.965257+0000","last_undegraded":"2024-09-22T03:37:16.965257+0000","last_fullsized":"2024-09-22T03:37:16.965257+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":768,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"106'720","reported_seq":769,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.858438+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:37:16.858438+0000","last_peered":"2024-09-22T03:37:16.858438+0000","last_clean":"2024-09-22T03:37:16.858438+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:37:16.858438+0000","last_undegraded":"2024-09-22T03:37:16.858438+0000","last_fullsized":"2024-09-22T03:37:16.858438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":427,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.817067+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:16.817067+0000","last_peered":"2024-09-22T03:37:16.817067+0000","last_clean":"2024-09-22T03:37:16.817067+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:16.817067+0000","last_undegraded":"2024-09-22T03:37:16.817067+0000","last_fullsized":"2024-09-22T03:37:16.817067+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":293,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.856727+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:16.856727+0000","last_peered":"2024-09-22T03:37:16.856727+0000","last_clean":"2024-09-22T03:37:16.856727+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:16.856727+0000","last_undegraded":"2024-09-22T03:37:16.856727+0000","last_fullsized":"2024-09-22T03:37:16.856727+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"108'791","reported_seq":840,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.981897+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:37:16.981897+0000","last_peered":"2024-09-22T03:37:16.981897+0000","last_clean":"2024-09-22T03:37:16.981897+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:37:16.981897+0000","last_undegraded":"2024-09-22T03:37:16.981897+0000","last_fullsized":"2024-09-22T03:37:16.981897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.860371+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:16.860371+0000","last_peered":"2024-09-22T03:37:16.860371+0000","last_clean":"2024-09-22T03:37:16.860371+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:16.860371+0000","last_undegraded":"2024-09-22T03:37:16.860371+0000","last_fullsized":"2024-09-22T03:37:16.860371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"106'776","reported_seq":824,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.818647+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:37:16.818647+0000","last_peered":"2024-09-22T03:37:16.818647+0000","last_clean":"2024-09-22T03:37:16.818647+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:37:16.818647+0000","last_undegraded":"2024-09-22T03:37:16.818647+0000","last_fullsized":"2024-09-22T03:37:16.818647+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.852841+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:16.852841+0000","last_peered":"2024-09-22T03:37:16.852841+0000","last_clean":"2024-09-22T03:37:16.852841+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:16.852841+0000","last_undegraded":"2024-09-22T03:37:16.852841+0000","last_fullsized":"2024-09-22T03:37:16.852841+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"106'712","reported_seq":760,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.857910+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:37:16.857910+0000","last_peered":"2024-09-22T03:37:16.857910+0000","last_clean":"2024-09-22T03:37:16.857910+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:37:16.857910+0000","last_undegraded":"2024-09-22T03:37:16.857910+0000","last_fullsized":"2024-09-22T03:37:16.857910+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5832704,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":294,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.852166+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:16.852166+0000","last_peered":"2024-09-22T03:37:16.852166+0000","last_clean":"2024-09-22T03:37:16.852166+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:16.852166+0000","last_undegraded":"2024-09-22T03:37:16.852166+0000","last_fullsized":"2024-09-22T03:37:16.852166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"108'807","reported_seq":856,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.877209+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:37:16.877209+0000","last_peered":"2024-09-22T03:37:16.877209+0000","last_clean":"2024-09-22T03:37:16.877209+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:37:16.877209+0000","last_undegraded":"2024-09-22T03:37:16.877209+0000","last_fullsized":"2024-09-22T03:37:16.877209+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.894088+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:16.894088+0000","last_peered":"2024-09-22T03:37:16.894088+0000","last_clean":"2024-09-22T03:37:16.894088+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:16.894088+0000","last_undegraded":"2024-09-22T03:37:16.894088+0000","last_fullsized":"2024-09-22T03:37:16.894088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"107'720","reported_seq":768,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.856187+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:37:16.856187+0000","last_peered":"2024-09-22T03:37:16.856187+0000","last_clean":"2024-09-22T03:37:16.856187+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:37:16.856187+0000","last_undegraded":"2024-09-22T03:37:16.856187+0000","last_fullsized":"2024-09-22T03:37:16.856187+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":720,"num_write_kb":5760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.820375+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:16.820375+0000","last_peered":"2024-09-22T03:37:16.820375+0000","last_clean":"2024-09-22T03:37:16.820375+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:16.820375+0000","last_undegraded":"2024-09-22T03:37:16.820375+0000","last_fullsized":"2024-09-22T03:37:16.820375+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"106'680","reported_seq":728,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.862340+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:37:16.862340+0000","last_peered":"2024-09-22T03:37:16.862340+0000","last_clean":"2024-09-22T03:37:16.862340+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:37:16.862340+0000","last_undegraded":"2024-09-22T03:37:16.862340+0000","last_fullsized":"2024-09-22T03:37:16.862340+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.894273+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:16.894273+0000","last_peered":"2024-09-22T03:37:16.894273+0000","last_clean":"2024-09-22T03:37:16.894273+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:16.894273+0000","last_undegraded":"2024-09-22T03:37:16.894273+0000","last_fullsized":"2024-09-22T03:37:16.894273+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"107'592","reported_seq":640,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.855975+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:37:16.855975+0000","last_peered":"2024-09-22T03:37:16.855975+0000","last_clean":"2024-09-22T03:37:16.855975+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:37:16.855975+0000","last_undegraded":"2024-09-22T03:37:16.855975+0000","last_fullsized":"2024-09-22T03:37:16.855975+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-22T03:37:16.858723+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:16.858723+0000","last_peered":"2024-09-22T03:37:16.858723+0000","last_clean":"2024-09-22T03:37:16.858723+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:16.858723+0000","last_undegraded":"2024-09-22T03:37:16.858723+0000","last_fullsized":"2024-09-22T03:37:16.858723+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":96542720,"num_objects":1474,"num_object_clones":0,"num_object_copies":4422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11785,"num_write_kb":94280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213601562624,"internally_reserved":0,"allocated":609931264,"data_stored":609931264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11781,"ondisk_log_size":11781,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213601570816,"internally_reserved":0,"allocated":609923072,"data_stored":609923072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":115264,"kb_used_data":115264,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52182464,"statfs":{"total":53552873472,"available":53434843136,"internally_reserved":0,"allocated":118030336,"data_stored":118030336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":172576,"kb_used_data":172576,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52125152,"statfs":{"total":53552873472,"available":53376155648,"internally_reserved":0,"allocated":176717824,"data_stored":176717824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":147080,"kb_used_data":147080,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52150648,"statfs":{"total":53552873472,"available":53402263552,"internally_reserved":0,"allocated":150609920,"data_stored":150609920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":156908,"kb_used_data":156908,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52140820,"statfs":{"total":53552873472,"available":53392199680,"internally_reserved":0,"allocated":160673792,"data_stored":160673792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53391048704,"internally_reserved":0,"allocated":161824768,"data_stored":161824768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53401284608,"internally_reserved":0,"allocated":151588864,"data_stored":151588864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53375164416,"internally_reserved":0,"allocated":177709056,"data_stored":177709056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53391040512,"internally_reserved":0,"allocated":161832960,"data_stored":161832960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53401284608,"internally_reserved":0,"allocated":151588864,"data_stored":151588864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53375164416,"internally_reserved":0,"allocated":177709056,"data_stored":177709056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53434073088,"internally_reserved":0,"allocated":118800384,"data_stored":118800384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:18.757 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:37:18.758 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:37:18.758 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:37:18.758 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:37:18.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:18.912+0000 7fe4b5d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:18.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:18.920+0000 7fe4b5d97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:18.936 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:18.936 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:18.936 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:18.936 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:18.936 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:18.936 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:18.936 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:18.937 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:18.937 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:18.937 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:18.937 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:18.937 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:18.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:18.939 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:18.939 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:37:19.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:19.051+0000 7f6d1d02c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:19.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:19.059+0000 7f6d1d02c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:19.075 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:19.076 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:19.076 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:19.076 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:19.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:19.079 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:19.079 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:37:19.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:19.196+0000 7febabf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:19.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:19.207+0000 7febabf57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:19.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout: 90 16 12029 12013 1.0427 0.71875 0.239863 0.119667 2024-09-22T03:37:19.246 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:19.246 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:19.247 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:19.247 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:19.247 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:19.247 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:19.247 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:19.247 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:19.247 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:19.248 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:19.248 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:19.248 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:19.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:19.251 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:19.251 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:37:19.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:19.358+0000 7fdacaa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:19.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:19.366+0000 7fdacaa6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:19.381 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:19.381 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:19.381 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:19.381 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:19.381 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:19.381 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:19.381 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:19.382 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:19.382 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:19.382 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:19.382 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:19.382 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:19.385 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:19.385 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:20.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total time run: 90.1906 2024-09-22T03:37:20.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Total writes made: 12029 2024-09-22T03:37:20.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Write size: 8192 2024-09-22T03:37:20.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Object size: 65536 2024-09-22T03:37:20.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Bandwidth (MB/sec): 1.04198 2024-09-22T03:37:20.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Bandwidth: 0.100002 2024-09-22T03:37:20.235 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max bandwidth (MB/sec): 1.32031 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min bandwidth (MB/sec): 0.71875 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average IOPS: 133 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev IOPS: 12.8003 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max IOPS: 169 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min IOPS: 92 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Average Latency(s): 0.119828 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Stddev Latency(s): 0.0676096 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Max latency(s): 0.55784 2024-09-22T03:37:20.236 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Min latency(s): 0.0145288 2024-09-22T03:37:20.263 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Cleaning up (deleting benchmark objects) 2024-09-22T03:37:20.697 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:37:20.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:20.847+0000 7f149f223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:20.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:20.853+0000 7f149f223640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:23.134 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:37:23.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:23.288+0000 7f0d6ecd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:23.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:23.294+0000 7f0d6ecd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:24.386 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:37:24.386 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:37:24.386 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-22T03:37:24.386 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-22T03:37:24.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:24.537+0000 7f1df2d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:24.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:24.552+0000 7f1df2d57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:24.642 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:24.642 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:24.642 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:24.643 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:24.646 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:24.647 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:24.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-22T03:37:24.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:24.805+0000 7fa9c7b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:24.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:24.816+0000 7fa9c7b17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:24.893 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:24.893 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:24.894 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:24.898 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:24.898 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:24.898 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-22T03:37:25.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:25.065+0000 7f2243f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:25.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:25.076+0000 7f2243f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:25.105 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:25.105 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:25.105 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:25.106 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:25.107 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:25.110 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:25.111 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:25.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-22T03:37:25.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:25.272+0000 7fd733f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:25.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:25.283+0000 7fd733f57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:25.306 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:config help 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:config show 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:deep_scrub 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_ops 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:dump_historic_slow_ops 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:dump_metrics [] 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:dump_ops_in_flight 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:dump_pgstate_history 2024-09-22T03:37:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:Error EINVAL: invalid command 2024-09-22T03:37:25.310 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-22T03:37:25.311 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-22T03:37:25.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:37:25.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:25.804+0000 7fd182f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:25.810+0000 7fd182f97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:28.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:37:28.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:28.137+0000 7f140b8ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:28.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:28.146+0000 7f140b8ce640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:30.312 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-22T03:37:30.312 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00 2024-09-22T03:37:30.313 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:30.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T03:37:30.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:30.472+0000 7f99b2ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:30.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:30.490+0000 7f99b2ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:30.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:30.537+0000 7fd33266c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:30.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:30.543+0000 7fd33266c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:30.789 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:30.804 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-09-22T03:37:30.096570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35299915,"num_objects":535,"num_object_clones":0,"num_object_copies":1605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":13143,"num_write_kb":98129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13088,"ondisk_log_size":13088,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":277588,"kb_used_data":277588,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208913324,"statfs":{"total":214211493888,"available":213927243776,"internally_reserved":0,"allocated":284250112,"data_stored":284250112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-61833192,"num_objects":-943,"num_object_clones":0,"num_object_copies":-2829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1219,"num_write_kb":1953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002277"},"pg_stats":[{"pgid":"10.7","version":"109'761","reported_seq":813,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.650132+0000","last_change":"2024-09-22T03:35:48.263412+0000","last_active":"2024-09-22T03:37:26.650132+0000","last_peered":"2024-09-22T03:37:26.650132+0000","last_clean":"2024-09-22T03:37:26.650132+0000","last_became_active":"2024-09-22T03:35:48.225002+0000","last_became_peered":"2024-09-22T03:35:48.225002+0000","last_unstale":"2024-09-22T03:37:26.650132+0000","last_undegraded":"2024-09-22T03:37:26.650132+0000","last_fullsized":"2024-09-22T03:37:26.650132+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":762,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"109'835","reported_seq":887,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.933687+0000","last_change":"2024-09-22T03:35:48.258153+0000","last_active":"2024-09-22T03:37:26.933687+0000","last_peered":"2024-09-22T03:37:26.933687+0000","last_clean":"2024-09-22T03:37:26.933687+0000","last_became_active":"2024-09-22T03:35:48.249385+0000","last_became_peered":"2024-09-22T03:35:48.249385+0000","last_unstale":"2024-09-22T03:37:26.933687+0000","last_undegraded":"2024-09-22T03:37:26.933687+0000","last_fullsized":"2024-09-22T03:37:26.933687+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":836,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"109'974","reported_seq":1026,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.809227+0000","last_change":"2024-09-22T03:35:48.271552+0000","last_active":"2024-09-22T03:37:26.809227+0000","last_peered":"2024-09-22T03:37:26.809227+0000","last_clean":"2024-09-22T03:37:26.809227+0000","last_became_active":"2024-09-22T03:35:48.245541+0000","last_became_peered":"2024-09-22T03:35:48.245541+0000","last_unstale":"2024-09-22T03:37:26.809227+0000","last_undegraded":"2024-09-22T03:37:26.809227+0000","last_fullsized":"2024-09-22T03:37:26.809227+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":975,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.4","version":"109'875","reported_seq":927,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.921677+0000","last_change":"2024-09-22T03:35:48.239873+0000","last_active":"2024-09-22T03:37:26.921677+0000","last_peered":"2024-09-22T03:37:26.921677+0000","last_clean":"2024-09-22T03:37:26.921677+0000","last_became_active":"2024-09-22T03:35:48.226980+0000","last_became_peered":"2024-09-22T03:35:48.226980+0000","last_unstale":"2024-09-22T03:37:26.921677+0000","last_undegraded":"2024-09-22T03:37:26.921677+0000","last_fullsized":"2024-09-22T03:37:26.921677+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2752512,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":876,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"109'750","reported_seq":802,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.913844+0000","last_change":"2024-09-22T03:35:48.322981+0000","last_active":"2024-09-22T03:37:26.913844+0000","last_peered":"2024-09-22T03:37:26.913844+0000","last_clean":"2024-09-22T03:37:26.913844+0000","last_became_active":"2024-09-22T03:35:48.317379+0000","last_became_peered":"2024-09-22T03:35:48.317379+0000","last_unstale":"2024-09-22T03:37:26.913844+0000","last_undegraded":"2024-09-22T03:37:26.913844+0000","last_fullsized":"2024-09-22T03:37:26.913844+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751,"num_write_kb":5568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"109'797","reported_seq":849,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.847948+0000","last_change":"2024-09-22T03:35:48.326496+0000","last_active":"2024-09-22T03:37:26.847948+0000","last_peered":"2024-09-22T03:37:26.847948+0000","last_clean":"2024-09-22T03:37:26.847948+0000","last_became_active":"2024-09-22T03:35:48.318998+0000","last_became_peered":"2024-09-22T03:35:48.318998+0000","last_unstale":"2024-09-22T03:37:26.847948+0000","last_undegraded":"2024-09-22T03:37:26.847948+0000","last_fullsized":"2024-09-22T03:37:26.847948+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":798,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"109'740","reported_seq":792,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.781345+0000","last_change":"2024-09-22T03:35:48.368784+0000","last_active":"2024-09-22T03:37:26.781345+0000","last_peered":"2024-09-22T03:37:26.781345+0000","last_clean":"2024-09-22T03:37:26.781345+0000","last_became_active":"2024-09-22T03:35:48.364340+0000","last_became_peered":"2024-09-22T03:35:48.364340+0000","last_unstale":"2024-09-22T03:37:26.781345+0000","last_undegraded":"2024-09-22T03:37:26.781345+0000","last_fullsized":"2024-09-22T03:37:26.781345+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":741,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"109'860","reported_seq":912,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.731356+0000","last_change":"2024-09-22T03:35:48.276884+0000","last_active":"2024-09-22T03:37:26.731356+0000","last_peered":"2024-09-22T03:37:26.731356+0000","last_clean":"2024-09-22T03:37:26.731356+0000","last_became_active":"2024-09-22T03:35:48.270481+0000","last_became_peered":"2024-09-22T03:35:48.270481+0000","last_unstale":"2024-09-22T03:37:26.731356+0000","last_undegraded":"2024-09-22T03:37:26.731356+0000","last_fullsized":"2024-09-22T03:37:26.731356+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":861,"num_write_kb":6440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"109'803","reported_seq":855,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.808269+0000","last_change":"2024-09-22T03:35:48.253458+0000","last_active":"2024-09-22T03:37:26.808269+0000","last_peered":"2024-09-22T03:37:26.808269+0000","last_clean":"2024-09-22T03:37:26.808269+0000","last_became_active":"2024-09-22T03:35:48.245014+0000","last_became_peered":"2024-09-22T03:35:48.245014+0000","last_unstale":"2024-09-22T03:37:26.808269+0000","last_undegraded":"2024-09-22T03:37:26.808269+0000","last_fullsized":"2024-09-22T03:37:26.808269+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":430,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.842786+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:17.842786+0000","last_peered":"2024-09-22T03:37:17.842786+0000","last_clean":"2024-09-22T03:37:17.842786+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:17.842786+0000","last_undegraded":"2024-09-22T03:37:17.842786+0000","last_fullsized":"2024-09-22T03:37:17.842786+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":296,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.830660+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:17.830660+0000","last_peered":"2024-09-22T03:37:17.830660+0000","last_clean":"2024-09-22T03:37:17.830660+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:17.830660+0000","last_undegraded":"2024-09-22T03:37:17.830660+0000","last_fullsized":"2024-09-22T03:37:17.830660+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"109'883","reported_seq":935,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.826670+0000","last_change":"2024-09-22T03:35:48.356573+0000","last_active":"2024-09-22T03:37:26.826670+0000","last_peered":"2024-09-22T03:37:26.826670+0000","last_clean":"2024-09-22T03:37:26.826670+0000","last_became_active":"2024-09-22T03:35:48.352884+0000","last_became_peered":"2024-09-22T03:35:48.352884+0000","last_unstale":"2024-09-22T03:37:26.826670+0000","last_undegraded":"2024-09-22T03:37:26.826670+0000","last_fullsized":"2024-09-22T03:37:26.826670+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":884,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":295,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.862407+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:17.862407+0000","last_peered":"2024-09-22T03:37:17.862407+0000","last_clean":"2024-09-22T03:37:17.862407+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:17.862407+0000","last_undegraded":"2024-09-22T03:37:17.862407+0000","last_fullsized":"2024-09-22T03:37:17.862407+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"109'854","reported_seq":906,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.859552+0000","last_change":"2024-09-22T03:35:48.313746+0000","last_active":"2024-09-22T03:37:26.859552+0000","last_peered":"2024-09-22T03:37:26.859552+0000","last_clean":"2024-09-22T03:37:26.859552+0000","last_became_active":"2024-09-22T03:35:48.307939+0000","last_became_peered":"2024-09-22T03:35:48.307939+0000","last_unstale":"2024-09-22T03:37:26.859552+0000","last_undegraded":"2024-09-22T03:37:26.859552+0000","last_fullsized":"2024-09-22T03:37:26.859552+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":855,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":295,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.826905+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:17.826905+0000","last_peered":"2024-09-22T03:37:17.826905+0000","last_clean":"2024-09-22T03:37:17.826905+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:17.826905+0000","last_undegraded":"2024-09-22T03:37:17.826905+0000","last_fullsized":"2024-09-22T03:37:17.826905+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"109'796","reported_seq":848,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.962739+0000","last_change":"2024-09-22T03:35:48.294945+0000","last_active":"2024-09-22T03:37:26.962739+0000","last_peered":"2024-09-22T03:37:26.962739+0000","last_clean":"2024-09-22T03:37:26.962739+0000","last_became_active":"2024-09-22T03:35:48.288312+0000","last_became_peered":"2024-09-22T03:35:48.288312+0000","last_unstale":"2024-09-22T03:37:26.962739+0000","last_undegraded":"2024-09-22T03:37:26.962739+0000","last_fullsized":"2024-09-22T03:37:26.962739+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":797,"num_write_kb":5888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":297,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.857697+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:17.857697+0000","last_peered":"2024-09-22T03:37:17.857697+0000","last_clean":"2024-09-22T03:37:17.857697+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:17.857697+0000","last_undegraded":"2024-09-22T03:37:17.857697+0000","last_fullsized":"2024-09-22T03:37:17.857697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.c","version":"109'884","reported_seq":936,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.806859+0000","last_change":"2024-09-22T03:35:48.256504+0000","last_active":"2024-09-22T03:37:26.806859+0000","last_peered":"2024-09-22T03:37:26.806859+0000","last_clean":"2024-09-22T03:37:26.806859+0000","last_became_active":"2024-09-22T03:35:48.252213+0000","last_became_peered":"2024-09-22T03:35:48.252213+0000","last_unstale":"2024-09-22T03:37:26.806859+0000","last_undegraded":"2024-09-22T03:37:26.806859+0000","last_fullsized":"2024-09-22T03:37:26.806859+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":885,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":295,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.829663+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:17.829663+0000","last_peered":"2024-09-22T03:37:17.829663+0000","last_clean":"2024-09-22T03:37:17.829663+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:17.829663+0000","last_undegraded":"2024-09-22T03:37:17.829663+0000","last_fullsized":"2024-09-22T03:37:17.829663+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"109'777","reported_seq":830,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.713542+0000","last_change":"2024-09-22T03:35:48.354491+0000","last_active":"2024-09-22T03:37:26.713542+0000","last_peered":"2024-09-22T03:37:26.713542+0000","last_clean":"2024-09-22T03:37:26.713542+0000","last_became_active":"2024-09-22T03:35:48.350449+0000","last_became_peered":"2024-09-22T03:35:48.350449+0000","last_unstale":"2024-09-22T03:37:26.713542+0000","last_undegraded":"2024-09-22T03:37:26.713542+0000","last_fullsized":"2024-09-22T03:37:26.713542+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162712,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":778,"num_write_kb":5761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":295,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.845768+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:17.845768+0000","last_peered":"2024-09-22T03:37:17.845768+0000","last_clean":"2024-09-22T03:37:17.845768+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:17.845768+0000","last_undegraded":"2024-09-22T03:37:17.845768+0000","last_fullsized":"2024-09-22T03:37:17.845768+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"109'743","reported_seq":795,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.742759+0000","last_change":"2024-09-22T03:35:48.271690+0000","last_active":"2024-09-22T03:37:26.742759+0000","last_peered":"2024-09-22T03:37:26.742759+0000","last_clean":"2024-09-22T03:37:26.742759+0000","last_became_active":"2024-09-22T03:35:48.267508+0000","last_became_peered":"2024-09-22T03:35:48.267508+0000","last_unstale":"2024-09-22T03:37:26.742759+0000","last_undegraded":"2024-09-22T03:37:26.742759+0000","last_fullsized":"2024-09-22T03:37:26.742759+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":295,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.827922+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:17.827922+0000","last_peered":"2024-09-22T03:37:17.827922+0000","last_clean":"2024-09-22T03:37:17.827922+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:17.827922+0000","last_undegraded":"2024-09-22T03:37:17.827922+0000","last_fullsized":"2024-09-22T03:37:17.827922+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"109'656","reported_seq":708,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:26.824889+0000","last_change":"2024-09-22T03:35:48.303481+0000","last_active":"2024-09-22T03:37:26.824889+0000","last_peered":"2024-09-22T03:37:26.824889+0000","last_clean":"2024-09-22T03:37:26.824889+0000","last_became_active":"2024-09-22T03:35:48.297512+0000","last_became_peered":"2024-09-22T03:35:48.297512+0000","last_unstale":"2024-09-22T03:37:26.824889+0000","last_undegraded":"2024-09-22T03:37:26.824889+0000","last_fullsized":"2024-09-22T03:37:26.824889+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:35:47.133123+0000","last_clean_scrub_stamp":"2024-09-22T03:35:47.133123+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":295,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T03:37:17.831114+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:17.831114+0000","last_peered":"2024-09-22T03:37:17.831114+0000","last_clean":"2024-09-22T03:37:17.831114+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:17.831114+0000","last_undegraded":"2024-09-22T03:37:17.831114+0000","last_fullsized":"2024-09-22T03:37:17.831114+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":34709528,"num_objects":531,"num_object_clones":0,"num_object_copies":1593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13004,"num_write_kb":96233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213968896000,"internally_reserved":0,"allocated":242597888,"data_stored":242597888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12988,"ondisk_log_size":12988,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":213968896000,"internally_reserved":0,"allocated":242597888,"data_stored":242597888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53505069056,"internally_reserved":0,"allocated":47804416,"data_stored":47804416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":55376,"kb_used_data":55376,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52242352,"statfs":{"total":53552873472,"available":53496168448,"internally_reserved":0,"allocated":56705024,"data_stored":56705024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":80176,"kb_used_data":80176,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52217552,"statfs":{"total":53552873472,"available":53470773248,"internally_reserved":0,"allocated":82100224,"data_stored":82100224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":69904,"kb_used_data":69904,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52227824,"statfs":{"total":53552873472,"available":53481291776,"internally_reserved":0,"allocated":71581696,"data_stored":71581696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":72132,"kb_used_data":72132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52225596,"statfs":{"total":53552873472,"available":53479010304,"internally_reserved":0,"allocated":73863168,"data_stored":73863168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53505069056,"internally_reserved":0,"allocated":47804416,"data_stored":47804416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53489311744,"internally_reserved":0,"allocated":63561728,"data_stored":63561728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53492568064,"internally_reserved":0,"allocated":60305408,"data_stored":60305408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53481816064,"internally_reserved":0,"allocated":71057408,"data_stored":71057408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53505200128,"internally_reserved":0,"allocated":47673344,"data_stored":47673344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":53552873472,"available":53489311744,"internally_reserved":0,"allocated":63561728,"data_stored":63561728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":53552873472,"available":53492568064,"internally_reserved":0,"allocated":60305408,"data_stored":60305408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":53552873472,"available":53481816064,"internally_reserved":0,"allocated":71057408,"data_stored":71057408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":53552873472,"available":53505200128,"internally_reserved":0,"allocated":47673344,"data_stored":47673344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:31.280 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Removed 1504 objects 2024-09-22T03:37:31.281 INFO:tasks.radosbench.radosbench.0.smithi129.stdout:Clean up completed and total clean up time :11.0171 2024-09-22T03:37:32.672 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T03:37:32.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:32.785+0000 7f2a6e84f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:32.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:32.789+0000 7f2a6e84f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:32.885 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_7 2024-09-22T03:37:32.885 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_7 unique_pool_7 --yes-i-really-really-mean-it 2024-09-22T03:37:32.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:32.983+0000 7f78c6197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:32.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:32.994+0000 7f78c6197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:33.837 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'unique_pool_7' removed 2024-09-22T03:37:33.854 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-22T03:37:33.865 INFO:tasks.thrashosds:joining thrashosds 2024-09-22T03:37:35.806 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-22T03:37:35.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:35.914+0000 7fc1eeace640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:35.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:35.919+0000 7fc1eeace640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:35.933 INFO:teuthology.orchestra.run.smithi129.stdout:{} 2024-09-22T03:37:35.933 INFO:teuthology.orchestra.run.smithi129.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T03:37:35.943 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T03:37:36.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:36.042+0000 7fc4fcd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:36.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:36.051+0000 7fc4fcd57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:36.842 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T03:37:36.860 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-22T03:37:36.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:36.959+0000 7fa7aef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:36.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:36.967+0000 7fa7aef97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:37.847 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-22T03:37:37.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T03:37:37.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:37.962+0000 7f7f10117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:37.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:37.970+0000 7f7f10117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:38.852 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T03:37:38.870 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-22T03:37:38.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:38.970+0000 7f4cbad17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:38.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:38.979+0000 7f4cbad17640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-22T03:37:39.885 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T03:37:40.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:39.996+0000 7f33fab97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:40.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:40.004+0000 7f33fab97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T03:37:40.892 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-22T03:37:41.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:40.995+0000 7fc5e4957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:41.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:41.004+0000 7fc5e4957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:41.877 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-22T03:37:41.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T03:37:42.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:42.001+0000 7ff774557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:42.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:42.010+0000 7ff774557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:42.885 INFO:teuthology.orchestra.run.smithi129.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T03:37:42.901 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-22T03:37:43.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:43.005+0000 7f430710e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:43.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:43.013+0000 7f430710e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:43.886 INFO:teuthology.orchestra.run.smithi129.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-22T03:37:43.904 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-22T03:37:43.904 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T03:37:43.904 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:37:44.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.006+0000 7f3278397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.012+0000 7f3278397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.189 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:44.189 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":119,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:37:43.884488+0000","last_up_change":"2024-09-22T03:23:01.787205+0000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:23:01.973068+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:23:05.895212+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":27947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":27947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":27947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":27947901}]},"public_addr":"172.21.15.129:6800/27947901","cluster_addr":"172.21.15.129:6801/27947901","heartbeat_back_addr":"172.21.15.129:6805/27947901","heartbeat_front_addr":"172.21.15.129:6802/27947901","state":["exists","up"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2887884686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2887884686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2887884686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2887884686}]},"public_addr":"172.21.15.129:6803/2887884686","cluster_addr":"172.21.15.129:6804/2887884686","heartbeat_back_addr":"172.21.15.129:6806/2887884686","heartbeat_front_addr":"172.21.15.129:6807/2887884686","state":["exists","up"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3028982311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":3028982311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3028982311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3028982311}]},"public_addr":"172.21.15.144:6804/3028982311","cluster_addr":"172.21.15.144:6803/3028982311","heartbeat_back_addr":"172.21.15.144:6814/3028982311","heartbeat_front_addr":"172.21.15.144:6815/3028982311","state":["exists","up"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1383918015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":1383918015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1383918015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":1383918015}]},"public_addr":"172.21.15.144:6806/1383918015","cluster_addr":"172.21.15.144:6807/1383918015","heartbeat_back_addr":"172.21.15.144:6808/1383918015","heartbeat_front_addr":"172.21.15.144:6805/1383918015","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:37:44.202 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T03:37:44.202 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:37:44.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.300+0000 7fe1ea26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.306+0000 7fe1ea26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.486 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:44.486 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":119,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:37:43.884488+0000","last_up_change":"2024-09-22T03:23:01.787205+0000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:23:01.973068+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:23:05.895212+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":27947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":27947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":27947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":27947901}]},"public_addr":"172.21.15.129:6800/27947901","cluster_addr":"172.21.15.129:6801/27947901","heartbeat_back_addr":"172.21.15.129:6805/27947901","heartbeat_front_addr":"172.21.15.129:6802/27947901","state":["exists","up"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2887884686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2887884686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2887884686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2887884686}]},"public_addr":"172.21.15.129:6803/2887884686","cluster_addr":"172.21.15.129:6804/2887884686","heartbeat_back_addr":"172.21.15.129:6806/2887884686","heartbeat_front_addr":"172.21.15.129:6807/2887884686","state":["exists","up"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3028982311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":3028982311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3028982311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3028982311}]},"public_addr":"172.21.15.144:6804/3028982311","cluster_addr":"172.21.15.144:6803/3028982311","heartbeat_back_addr":"172.21.15.144:6814/3028982311","heartbeat_front_addr":"172.21.15.144:6815/3028982311","state":["exists","up"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1383918015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":1383918015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1383918015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":1383918015}]},"public_addr":"172.21.15.144:6806/1383918015","cluster_addr":"172.21.15.144:6807/1383918015","heartbeat_back_addr":"172.21.15.144:6808/1383918015","heartbeat_front_addr":"172.21.15.144:6805/1383918015","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:37:44.500 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T03:37:44.500 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T03:37:44.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T03:37:44.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T03:37:44.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.611+0000 7f1a8bf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.611+0000 7f6868a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.611+0000 7f6707557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.611+0000 7fb2612c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.619+0000 7fb2612c3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.620+0000 7f6868a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.620+0000 7f1a8bf97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.620+0000 7f6707557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.656 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:44.656 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:37:44.657 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:44.658 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:37:44.658 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:44.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:37:44.659 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:44.660 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:37:44.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.807+0000 7ff516597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.807+0000 7ff62eeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.807+0000 7f767c623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.807+0000 7f4aa6218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.816+0000 7ff62eeb2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.816+0000 7f767c623640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.816+0000 7f4aa6218640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:44.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:44.816+0000 7ff516597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:45.003 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:45.008 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:45.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.1 2024-09-22T03:37:45.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.2 2024-09-22T03:37:45.030 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:45.038 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:45.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.3 2024-09-22T03:37:45.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.0 2024-09-22T03:37:46.018 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:37:46.023 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:37:46.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:37:46.052 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:37:46.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.132+0000 7f46e7b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.146+0000 7f46e7b57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.175+0000 7f16bd117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.176+0000 7fedd206c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.183+0000 7f16bd117640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.184+0000 7fedd206c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.179+0000 7f19d3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:46.190+0000 7f19d3e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:46.354 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:46.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.1 2024-09-22T03:37:46.377 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:46.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.0 2024-09-22T03:37:46.397 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:46.405 INFO:teuthology.orchestra.run.smithi129.stdout:38654706543 2024-09-22T03:37:46.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.3 2024-09-22T03:37:46.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706543 for osd.2 2024-09-22T03:37:47.368 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:37:47.392 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:37:47.412 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:37:47.420 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:37:47.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.490+0000 7fa703ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.501+0000 7fa703ad7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.527+0000 7fec48c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.537+0000 7fec48c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.534+0000 7fb852957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.548+0000 7fb852957640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.572+0000 7f69b984f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.580+0000 7f69b984f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.698 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:47.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706546 for osd.1 2024-09-22T03:37:47.712 DEBUG:teuthology.parallel:result is None 2024-09-22T03:37:47.729 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:47.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706546 for osd.0 2024-09-22T03:37:47.743 DEBUG:teuthology.parallel:result is None 2024-09-22T03:37:47.749 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:47.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706546 for osd.3 2024-09-22T03:37:47.763 DEBUG:teuthology.parallel:result is None 2024-09-22T03:37:47.781 INFO:teuthology.orchestra.run.smithi129.stdout:38654706546 2024-09-22T03:37:47.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654706546 got 38654706546 for osd.2 2024-09-22T03:37:47.795 DEBUG:teuthology.parallel:result is None 2024-09-22T03:37:47.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T03:37:47.795 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:47.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.941+0000 7f9b2b157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:47.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:47.948+0000 7f9b2b157640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:48.126 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:48.126 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:48.138 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-09-22T03:37:48.100417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":7716,"kb_used_data":7716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209183196,"statfs":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.222530"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":460,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905490+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:43.905490+0000","last_peered":"2024-09-22T03:37:43.905490+0000","last_clean":"2024-09-22T03:37:43.905490+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:43.905490+0000","last_undegraded":"2024-09-22T03:37:43.905490+0000","last_fullsized":"2024-09-22T03:37:43.905490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":326,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908495+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:43.908495+0000","last_peered":"2024-09-22T03:37:43.908495+0000","last_clean":"2024-09-22T03:37:43.908495+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:43.908495+0000","last_undegraded":"2024-09-22T03:37:43.908495+0000","last_fullsized":"2024-09-22T03:37:43.908495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907470+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:43.907470+0000","last_peered":"2024-09-22T03:37:43.907470+0000","last_clean":"2024-09-22T03:37:43.907470+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:43.907470+0000","last_undegraded":"2024-09-22T03:37:43.907470+0000","last_fullsized":"2024-09-22T03:37:43.907470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905774+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905774+0000","last_peered":"2024-09-22T03:37:43.905774+0000","last_clean":"2024-09-22T03:37:43.905774+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:43.905774+0000","last_undegraded":"2024-09-22T03:37:43.905774+0000","last_fullsized":"2024-09-22T03:37:43.905774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905582+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905582+0000","last_peered":"2024-09-22T03:37:43.905582+0000","last_clean":"2024-09-22T03:37:43.905582+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:43.905582+0000","last_undegraded":"2024-09-22T03:37:43.905582+0000","last_fullsized":"2024-09-22T03:37:43.905582+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903255+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.903255+0000","last_peered":"2024-09-22T03:37:43.903255+0000","last_clean":"2024-09-22T03:37:43.903255+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:43.903255+0000","last_undegraded":"2024-09-22T03:37:43.903255+0000","last_fullsized":"2024-09-22T03:37:43.903255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907847+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.907847+0000","last_peered":"2024-09-22T03:37:43.907847+0000","last_clean":"2024-09-22T03:37:43.907847+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:43.907847+0000","last_undegraded":"2024-09-22T03:37:43.907847+0000","last_fullsized":"2024-09-22T03:37:43.907847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903253+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:43.903253+0000","last_peered":"2024-09-22T03:37:43.903253+0000","last_clean":"2024-09-22T03:37:43.903253+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:43.903253+0000","last_undegraded":"2024-09-22T03:37:43.903253+0000","last_fullsized":"2024-09-22T03:37:43.903253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908284+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:43.908284+0000","last_peered":"2024-09-22T03:37:43.908284+0000","last_clean":"2024-09-22T03:37:43.908284+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:43.908284+0000","last_undegraded":"2024-09-22T03:37:43.908284+0000","last_fullsized":"2024-09-22T03:37:43.908284+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1884,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295844,"statfs":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2228,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295500,"statfs":{"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1324,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296404,"statfs":{"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2280,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295448,"statfs":{"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:48.139 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:48.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:48.238+0000 7f35d0797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:48.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:48.244+0000 7f35d0797640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:48.422 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:48.435 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-09-22T03:37:48.100417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":7716,"kb_used_data":7716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209183196,"statfs":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.222530"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":460,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905490+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:43.905490+0000","last_peered":"2024-09-22T03:37:43.905490+0000","last_clean":"2024-09-22T03:37:43.905490+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:43.905490+0000","last_undegraded":"2024-09-22T03:37:43.905490+0000","last_fullsized":"2024-09-22T03:37:43.905490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":326,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908495+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:43.908495+0000","last_peered":"2024-09-22T03:37:43.908495+0000","last_clean":"2024-09-22T03:37:43.908495+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:43.908495+0000","last_undegraded":"2024-09-22T03:37:43.908495+0000","last_fullsized":"2024-09-22T03:37:43.908495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907470+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:43.907470+0000","last_peered":"2024-09-22T03:37:43.907470+0000","last_clean":"2024-09-22T03:37:43.907470+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:43.907470+0000","last_undegraded":"2024-09-22T03:37:43.907470+0000","last_fullsized":"2024-09-22T03:37:43.907470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905774+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905774+0000","last_peered":"2024-09-22T03:37:43.905774+0000","last_clean":"2024-09-22T03:37:43.905774+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:43.905774+0000","last_undegraded":"2024-09-22T03:37:43.905774+0000","last_fullsized":"2024-09-22T03:37:43.905774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905582+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905582+0000","last_peered":"2024-09-22T03:37:43.905582+0000","last_clean":"2024-09-22T03:37:43.905582+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:43.905582+0000","last_undegraded":"2024-09-22T03:37:43.905582+0000","last_fullsized":"2024-09-22T03:37:43.905582+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903255+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.903255+0000","last_peered":"2024-09-22T03:37:43.903255+0000","last_clean":"2024-09-22T03:37:43.903255+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:43.903255+0000","last_undegraded":"2024-09-22T03:37:43.903255+0000","last_fullsized":"2024-09-22T03:37:43.903255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907847+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.907847+0000","last_peered":"2024-09-22T03:37:43.907847+0000","last_clean":"2024-09-22T03:37:43.907847+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:43.907847+0000","last_undegraded":"2024-09-22T03:37:43.907847+0000","last_fullsized":"2024-09-22T03:37:43.907847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903253+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:43.903253+0000","last_peered":"2024-09-22T03:37:43.903253+0000","last_clean":"2024-09-22T03:37:43.903253+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:43.903253+0000","last_undegraded":"2024-09-22T03:37:43.903253+0000","last_fullsized":"2024-09-22T03:37:43.903253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908284+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:43.908284+0000","last_peered":"2024-09-22T03:37:43.908284+0000","last_clean":"2024-09-22T03:37:43.908284+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:43.908284+0000","last_undegraded":"2024-09-22T03:37:43.908284+0000","last_fullsized":"2024-09-22T03:37:43.908284+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1884,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295844,"statfs":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2228,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295500,"statfs":{"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1324,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296404,"statfs":{"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2280,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295448,"statfs":{"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:48.435 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T03:37:48.436 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-22T03:37:48.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:48.534+0000 7f7cbf557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:48.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:48.543+0000 7f7cbf557640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:48.733 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-22T03:37:48.749 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T03:37:48.749 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:48.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:48.852+0000 7fa368717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:48.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:48.859+0000 7fa368717640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:49.033 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:49.045 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-09-22T03:37:48.100417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":7716,"kb_used_data":7716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209183196,"statfs":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.222530"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":460,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905490+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:43.905490+0000","last_peered":"2024-09-22T03:37:43.905490+0000","last_clean":"2024-09-22T03:37:43.905490+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:43.905490+0000","last_undegraded":"2024-09-22T03:37:43.905490+0000","last_fullsized":"2024-09-22T03:37:43.905490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":326,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908495+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:43.908495+0000","last_peered":"2024-09-22T03:37:43.908495+0000","last_clean":"2024-09-22T03:37:43.908495+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:43.908495+0000","last_undegraded":"2024-09-22T03:37:43.908495+0000","last_fullsized":"2024-09-22T03:37:43.908495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907470+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:43.907470+0000","last_peered":"2024-09-22T03:37:43.907470+0000","last_clean":"2024-09-22T03:37:43.907470+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:43.907470+0000","last_undegraded":"2024-09-22T03:37:43.907470+0000","last_fullsized":"2024-09-22T03:37:43.907470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905774+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905774+0000","last_peered":"2024-09-22T03:37:43.905774+0000","last_clean":"2024-09-22T03:37:43.905774+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:43.905774+0000","last_undegraded":"2024-09-22T03:37:43.905774+0000","last_fullsized":"2024-09-22T03:37:43.905774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905582+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905582+0000","last_peered":"2024-09-22T03:37:43.905582+0000","last_clean":"2024-09-22T03:37:43.905582+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:43.905582+0000","last_undegraded":"2024-09-22T03:37:43.905582+0000","last_fullsized":"2024-09-22T03:37:43.905582+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903255+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.903255+0000","last_peered":"2024-09-22T03:37:43.903255+0000","last_clean":"2024-09-22T03:37:43.903255+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:43.903255+0000","last_undegraded":"2024-09-22T03:37:43.903255+0000","last_fullsized":"2024-09-22T03:37:43.903255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907847+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.907847+0000","last_peered":"2024-09-22T03:37:43.907847+0000","last_clean":"2024-09-22T03:37:43.907847+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:43.907847+0000","last_undegraded":"2024-09-22T03:37:43.907847+0000","last_fullsized":"2024-09-22T03:37:43.907847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903253+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:43.903253+0000","last_peered":"2024-09-22T03:37:43.903253+0000","last_clean":"2024-09-22T03:37:43.903253+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:43.903253+0000","last_undegraded":"2024-09-22T03:37:43.903253+0000","last_fullsized":"2024-09-22T03:37:43.903253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908284+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:43.908284+0000","last_peered":"2024-09-22T03:37:43.908284+0000","last_clean":"2024-09-22T03:37:43.908284+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:43.908284+0000","last_undegraded":"2024-09-22T03:37:43.908284+0000","last_fullsized":"2024-09-22T03:37:43.908284+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1884,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295844,"statfs":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2228,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295500,"statfs":{"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1324,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296404,"statfs":{"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2280,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295448,"statfs":{"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:49.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:49.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:49.145+0000 7fa2a7cd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:49.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:49.152+0000 7fa2a7cd7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:49.328 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:49.340 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-09-22T03:37:48.100417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":7716,"kb_used_data":7716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209183196,"statfs":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.222530"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":460,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905490+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:43.905490+0000","last_peered":"2024-09-22T03:37:43.905490+0000","last_clean":"2024-09-22T03:37:43.905490+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:43.905490+0000","last_undegraded":"2024-09-22T03:37:43.905490+0000","last_fullsized":"2024-09-22T03:37:43.905490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":326,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908495+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:43.908495+0000","last_peered":"2024-09-22T03:37:43.908495+0000","last_clean":"2024-09-22T03:37:43.908495+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:43.908495+0000","last_undegraded":"2024-09-22T03:37:43.908495+0000","last_fullsized":"2024-09-22T03:37:43.908495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907470+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:43.907470+0000","last_peered":"2024-09-22T03:37:43.907470+0000","last_clean":"2024-09-22T03:37:43.907470+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:43.907470+0000","last_undegraded":"2024-09-22T03:37:43.907470+0000","last_fullsized":"2024-09-22T03:37:43.907470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905774+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905774+0000","last_peered":"2024-09-22T03:37:43.905774+0000","last_clean":"2024-09-22T03:37:43.905774+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:43.905774+0000","last_undegraded":"2024-09-22T03:37:43.905774+0000","last_fullsized":"2024-09-22T03:37:43.905774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905582+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905582+0000","last_peered":"2024-09-22T03:37:43.905582+0000","last_clean":"2024-09-22T03:37:43.905582+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:43.905582+0000","last_undegraded":"2024-09-22T03:37:43.905582+0000","last_fullsized":"2024-09-22T03:37:43.905582+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903255+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.903255+0000","last_peered":"2024-09-22T03:37:43.903255+0000","last_clean":"2024-09-22T03:37:43.903255+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:43.903255+0000","last_undegraded":"2024-09-22T03:37:43.903255+0000","last_fullsized":"2024-09-22T03:37:43.903255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907847+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.907847+0000","last_peered":"2024-09-22T03:37:43.907847+0000","last_clean":"2024-09-22T03:37:43.907847+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:43.907847+0000","last_undegraded":"2024-09-22T03:37:43.907847+0000","last_fullsized":"2024-09-22T03:37:43.907847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903253+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:43.903253+0000","last_peered":"2024-09-22T03:37:43.903253+0000","last_clean":"2024-09-22T03:37:43.903253+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:43.903253+0000","last_undegraded":"2024-09-22T03:37:43.903253+0000","last_fullsized":"2024-09-22T03:37:43.903253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908284+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:43.908284+0000","last_peered":"2024-09-22T03:37:43.908284+0000","last_clean":"2024-09-22T03:37:43.908284+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:43.908284+0000","last_undegraded":"2024-09-22T03:37:43.908284+0000","last_fullsized":"2024-09-22T03:37:43.908284+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1884,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295844,"statfs":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2228,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295500,"statfs":{"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1324,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296404,"statfs":{"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2280,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295448,"statfs":{"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:49.341 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T03:37:49.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:49.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:49.439+0000 7fc391757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:49.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:49.446+0000 7fc391757640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:49.621 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:49.633 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-09-22T03:37:48.100417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":7716,"kb_used_data":7716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209183196,"statfs":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.222530"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":460,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905490+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:43.905490+0000","last_peered":"2024-09-22T03:37:43.905490+0000","last_clean":"2024-09-22T03:37:43.905490+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:43.905490+0000","last_undegraded":"2024-09-22T03:37:43.905490+0000","last_fullsized":"2024-09-22T03:37:43.905490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":326,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908495+0000","last_change":"2024-09-22T03:23:09.844251+0000","last_active":"2024-09-22T03:37:43.908495+0000","last_peered":"2024-09-22T03:37:43.908495+0000","last_clean":"2024-09-22T03:37:43.908495+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:43.908495+0000","last_undegraded":"2024-09-22T03:37:43.908495+0000","last_fullsized":"2024-09-22T03:37:43.908495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907470+0000","last_change":"2024-09-22T03:23:09.844939+0000","last_active":"2024-09-22T03:37:43.907470+0000","last_peered":"2024-09-22T03:37:43.907470+0000","last_clean":"2024-09-22T03:37:43.907470+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:43.907470+0000","last_undegraded":"2024-09-22T03:37:43.907470+0000","last_fullsized":"2024-09-22T03:37:43.907470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905774+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905774+0000","last_peered":"2024-09-22T03:37:43.905774+0000","last_clean":"2024-09-22T03:37:43.905774+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:43.905774+0000","last_undegraded":"2024-09-22T03:37:43.905774+0000","last_fullsized":"2024-09-22T03:37:43.905774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905582+0000","last_change":"2024-09-22T03:23:09.841202+0000","last_active":"2024-09-22T03:37:43.905582+0000","last_peered":"2024-09-22T03:37:43.905582+0000","last_clean":"2024-09-22T03:37:43.905582+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:43.905582+0000","last_undegraded":"2024-09-22T03:37:43.905582+0000","last_fullsized":"2024-09-22T03:37:43.905582+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903255+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.903255+0000","last_peered":"2024-09-22T03:37:43.903255+0000","last_clean":"2024-09-22T03:37:43.903255+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:43.903255+0000","last_undegraded":"2024-09-22T03:37:43.903255+0000","last_fullsized":"2024-09-22T03:37:43.903255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907847+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.907847+0000","last_peered":"2024-09-22T03:37:43.907847+0000","last_clean":"2024-09-22T03:37:43.907847+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:43.907847+0000","last_undegraded":"2024-09-22T03:37:43.907847+0000","last_fullsized":"2024-09-22T03:37:43.907847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903253+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:43.903253+0000","last_peered":"2024-09-22T03:37:43.903253+0000","last_clean":"2024-09-22T03:37:43.903253+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:43.903253+0000","last_undegraded":"2024-09-22T03:37:43.903253+0000","last_fullsized":"2024-09-22T03:37:43.903253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.908284+0000","last_change":"2024-09-22T03:23:09.844208+0000","last_active":"2024-09-22T03:37:43.908284+0000","last_peered":"2024-09-22T03:37:43.908284+0000","last_clean":"2024-09-22T03:37:43.908284+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:43.908284+0000","last_undegraded":"2024-09-22T03:37:43.908284+0000","last_fullsized":"2024-09-22T03:37:43.908284+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1884,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295844,"statfs":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2228,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295500,"statfs":{"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1324,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296404,"statfs":{"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2280,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295448,"statfs":{"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:49.634 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:37:49.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:49.733+0000 7f8458597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:49.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:49.739+0000 7f8458597640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:49.915 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:49.915 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":119,"fsid":"7f5a2dae-b56a-4afe-bb9b-fb3f65651791","created":"2024-09-22T03:22:55.698211+0000","modified":"2024-09-22T03:37:43.884488+0000","last_up_change":"2024-09-22T03:23:01.787205+0000","last_in_change":"2024-09-22T03:22:57.794142+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:23:01.973068+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:23:05.895212+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b141605f-6391-41b8-b680-cb81bf703b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":27947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":27947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6805","nonce":27947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":27947901}]},"public_addr":"172.21.15.129:6800/27947901","cluster_addr":"172.21.15.129:6801/27947901","heartbeat_back_addr":"172.21.15.129:6805/27947901","heartbeat_front_addr":"172.21.15.129:6802/27947901","state":["exists","up"]},{"osd":1,"uuid":"a1a59185-9c42-4359-af1f-3dd8be188aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6803","nonce":2887884686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2887884686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2887884686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":2887884686}]},"public_addr":"172.21.15.129:6803/2887884686","cluster_addr":"172.21.15.129:6804/2887884686","heartbeat_back_addr":"172.21.15.129:6806/2887884686","heartbeat_front_addr":"172.21.15.129:6807/2887884686","state":["exists","up"]},{"osd":2,"uuid":"c1b83060-4307-4966-a545-5476767a17c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3028982311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6803","nonce":3028982311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3028982311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":3028982311}]},"public_addr":"172.21.15.144:6804/3028982311","cluster_addr":"172.21.15.144:6803/3028982311","heartbeat_back_addr":"172.21.15.144:6814/3028982311","heartbeat_front_addr":"172.21.15.144:6815/3028982311","state":["exists","up"]},{"osd":3,"uuid":"487e7a88-d3c1-469d-83b4-222cb8676d02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1383918015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":1383918015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1383918015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6805","nonce":1383918015}]},"public_addr":"172.21.15.144:6806/1383918015","cluster_addr":"172.21.15.144:6807/1383918015","heartbeat_back_addr":"172.21.15.144:6808/1383918015","heartbeat_front_addr":"172.21.15.144:6805/1383918015","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:37:50.930 INFO:tasks.ceph:Scrubbing osd.0 2024-09-22T03:37:50.930 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:37:51.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.041+0000 7f7d8bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.052+0000 7f7d8bb97640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.066 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-22T03:37:51.066 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "" 2024-09-22T03:37:51.066 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-22T03:37:51.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-22T03:37:51.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.176+0000 7ff167397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.183+0000 7ff167397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.361 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 0 to deep-scrub 2024-09-22T03:37:51.377 INFO:tasks.ceph:Scrubbing osd.1 2024-09-22T03:37:51.377 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:37:51.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.477+0000 7f23bb517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.487+0000 7f23bb517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.501 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-22T03:37:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "" 2024-09-22T03:37:51.501 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-22T03:37:51.512 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-22T03:37:51.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.610+0000 7efc26397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.618+0000 7efc26397640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.796 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 1 to deep-scrub 2024-09-22T03:37:51.813 INFO:tasks.ceph:Scrubbing osd.2 2024-09-22T03:37:51.813 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:37:51.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.912+0000 7f05d0a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:51.923+0000 7f05d0a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:51.939 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-22T03:37:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "" 2024-09-22T03:37:51.939 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-22T03:37:51.950 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-22T03:37:52.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.049+0000 7fdea5197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.056+0000 7fdea5197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.233 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 2 to deep-scrub 2024-09-22T03:37:52.249 INFO:tasks.ceph:Scrubbing osd.3 2024-09-22T03:37:52.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-22T03:37:52.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.348+0000 7fe619517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.359+0000 7fe619517640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.374 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-22T03:37:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "" 2024-09-22T03:37:52.375 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-22T03:37:52.385 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-22T03:37:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.483+0000 7f0673197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.491+0000 7f0673197640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.669 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 3 to deep-scrub 2024-09-22T03:37:52.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:37:52.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.781+0000 7ff8ce997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:37:52.788+0000 7ff8ce997640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:37:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:37:52.964 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:37:52.976 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":556,"stamp":"2024-09-22T03:37:52.101279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":7716,"kb_used_data":7716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209183196,"statfs":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.215772"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":460,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.905490+0000","last_change":"2024-09-22T03:23:03.846127+0000","last_active":"2024-09-22T03:37:43.905490+0000","last_peered":"2024-09-22T03:37:43.905490+0000","last_clean":"2024-09-22T03:37:43.905490+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:43.905490+0000","last_undegraded":"2024-09-22T03:37:43.905490+0000","last_fullsized":"2024-09-22T03:37:43.905490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:02.790903+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":328,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.798124+0000","last_change":"2024-09-22T03:37:51.817181+0000","last_active":"2024-09-22T03:37:51.798124+0000","last_peered":"2024-09-22T03:37:51.798124+0000","last_clean":"2024-09-22T03:37:51.798124+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:51.798124+0000","last_undegraded":"2024-09-22T03:37:51.798124+0000","last_fullsized":"2024-09-22T03:37:51.798124+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'1","last_scrub_stamp":"2024-09-22T03:37:51.817028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.817028+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.364056+0000","last_change":"2024-09-22T03:37:51.374295+0000","last_active":"2024-09-22T03:37:51.364056+0000","last_peered":"2024-09-22T03:37:51.364056+0000","last_clean":"2024-09-22T03:37:51.364056+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:51.364056+0000","last_undegraded":"2024-09-22T03:37:51.364056+0000","last_fullsized":"2024-09-22T03:37:51.364056+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:51.374201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.374201+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.796846+0000","last_change":"2024-09-22T03:37:51.807626+0000","last_active":"2024-09-22T03:37:51.796846+0000","last_peered":"2024-09-22T03:37:51.796846+0000","last_clean":"2024-09-22T03:37:51.796846+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:51.796846+0000","last_undegraded":"2024-09-22T03:37:51.796846+0000","last_fullsized":"2024-09-22T03:37:51.796846+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:51.807531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.807531+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":329,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.362807+0000","last_change":"2024-09-22T03:37:51.380801+0000","last_active":"2024-09-22T03:37:51.362807+0000","last_peered":"2024-09-22T03:37:51.362807+0000","last_clean":"2024-09-22T03:37:51.362807+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:51.362807+0000","last_undegraded":"2024-09-22T03:37:51.362807+0000","last_fullsized":"2024-09-22T03:37:51.362807+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2024-09-22T03:37:51.380676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.380676+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903255+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.903255+0000","last_peered":"2024-09-22T03:37:43.903255+0000","last_clean":"2024-09-22T03:37:43.903255+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:43.903255+0000","last_undegraded":"2024-09-22T03:37:43.903255+0000","last_fullsized":"2024-09-22T03:37:43.903255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.907847+0000","last_change":"2024-09-22T03:23:09.839455+0000","last_active":"2024-09-22T03:37:43.907847+0000","last_peered":"2024-09-22T03:37:43.907847+0000","last_clean":"2024-09-22T03:37:43.907847+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:43.907847+0000","last_undegraded":"2024-09-22T03:37:43.907847+0000","last_fullsized":"2024-09-22T03:37:43.907847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:43.903253+0000","last_change":"2024-09-22T03:23:09.839458+0000","last_active":"2024-09-22T03:37:43.903253+0000","last_peered":"2024-09-22T03:37:43.903253+0000","last_clean":"2024-09-22T03:37:43.903253+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:43.903253+0000","last_undegraded":"2024-09-22T03:37:43.903253+0000","last_fullsized":"2024-09-22T03:37:43.903253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:23:06.804798+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.798214+0000","last_change":"2024-09-22T03:37:51.813028+0000","last_active":"2024-09-22T03:37:51.798214+0000","last_peered":"2024-09-22T03:37:51.798214+0000","last_clean":"2024-09-22T03:37:51.798214+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:51.798214+0000","last_undegraded":"2024-09-22T03:37:51.798214+0000","last_fullsized":"2024-09-22T03:37:51.798214+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:51.812934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.812934+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1884,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295844,"statfs":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2228,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295500,"statfs":{"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1324,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296404,"statfs":{"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2280,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295448,"statfs":{"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:37:52.979 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-22T03:23:02.790903+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=23, tm_sec=2, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=37, tm_sec=49, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:37:52.979 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-22T03:23:06.804798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=23, tm_sec=6, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=37, tm_sec=49, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:37:52.979 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-22T03:23:06.804798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=23, tm_sec=6, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=37, tm_sec=49, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:37:52.979 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-22T03:23:06.804798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=23, tm_sec=6, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=37, tm_sec=49, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T03:37:52.980 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-22T03:38:12.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:38:13.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:38:13.092+0000 7f1b8a0b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:38:13.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:38:13.098+0000 7f1b8a0b2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:38:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-22T03:38:13.276 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-22T03:38:13.288 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-09-22T03:38:12.105446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":7716,"kb_used_data":7716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209183196,"statfs":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002527"},"pg_stats":[{"pgid":"1.0","version":"12'97","reported_seq":462,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:52.670022+0000","last_change":"2024-09-22T03:37:52.725483+0000","last_active":"2024-09-22T03:37:52.670022+0000","last_peered":"2024-09-22T03:37:52.670022+0000","last_clean":"2024-09-22T03:37:52.670022+0000","last_became_active":"2024-09-22T03:23:03.833723+0000","last_became_peered":"2024-09-22T03:23:03.833723+0000","last_unstale":"2024-09-22T03:37:52.670022+0000","last_undegraded":"2024-09-22T03:37:52.670022+0000","last_fullsized":"2024-09-22T03:37:52.670022+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'97","last_scrub_stamp":"2024-09-22T03:37:52.725290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:52.725290+0000","last_clean_scrub_stamp":"2024-09-22T03:23:02.790903+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":328,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.798124+0000","last_change":"2024-09-22T03:37:51.817181+0000","last_active":"2024-09-22T03:37:51.798124+0000","last_peered":"2024-09-22T03:37:51.798124+0000","last_clean":"2024-09-22T03:37:51.798124+0000","last_became_active":"2024-09-22T03:23:07.870886+0000","last_became_peered":"2024-09-22T03:23:07.870886+0000","last_unstale":"2024-09-22T03:37:51.798124+0000","last_undegraded":"2024-09-22T03:37:51.798124+0000","last_fullsized":"2024-09-22T03:37:51.798124+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'1","last_scrub_stamp":"2024-09-22T03:37:51.817028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.817028+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.364056+0000","last_change":"2024-09-22T03:37:51.374295+0000","last_active":"2024-09-22T03:37:51.364056+0000","last_peered":"2024-09-22T03:37:51.364056+0000","last_clean":"2024-09-22T03:37:51.364056+0000","last_became_active":"2024-09-22T03:23:07.873320+0000","last_became_peered":"2024-09-22T03:23:07.873320+0000","last_unstale":"2024-09-22T03:37:51.364056+0000","last_undegraded":"2024-09-22T03:37:51.364056+0000","last_fullsized":"2024-09-22T03:37:51.364056+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:51.374201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.374201+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.796846+0000","last_change":"2024-09-22T03:37:51.807626+0000","last_active":"2024-09-22T03:37:51.796846+0000","last_peered":"2024-09-22T03:37:51.796846+0000","last_clean":"2024-09-22T03:37:51.796846+0000","last_became_active":"2024-09-22T03:23:07.864822+0000","last_became_peered":"2024-09-22T03:23:07.864822+0000","last_unstale":"2024-09-22T03:37:51.796846+0000","last_undegraded":"2024-09-22T03:37:51.796846+0000","last_fullsized":"2024-09-22T03:37:51.796846+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:51.807531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.807531+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":329,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.362807+0000","last_change":"2024-09-22T03:37:51.380801+0000","last_active":"2024-09-22T03:37:51.362807+0000","last_peered":"2024-09-22T03:37:51.362807+0000","last_clean":"2024-09-22T03:37:51.362807+0000","last_became_active":"2024-09-22T03:23:07.889718+0000","last_became_peered":"2024-09-22T03:23:07.889718+0000","last_unstale":"2024-09-22T03:37:51.362807+0000","last_undegraded":"2024-09-22T03:37:51.362807+0000","last_fullsized":"2024-09-22T03:37:51.362807+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2024-09-22T03:37:51.380676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.380676+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:52.234898+0000","last_change":"2024-09-22T03:37:52.245033+0000","last_active":"2024-09-22T03:37:52.234898+0000","last_peered":"2024-09-22T03:37:52.234898+0000","last_clean":"2024-09-22T03:37:52.234898+0000","last_became_active":"2024-09-22T03:23:07.870641+0000","last_became_peered":"2024-09-22T03:23:07.870641+0000","last_unstale":"2024-09-22T03:37:52.234898+0000","last_undegraded":"2024-09-22T03:37:52.234898+0000","last_fullsized":"2024-09-22T03:37:52.234898+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:52.244878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:52.244878+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:52.671951+0000","last_change":"2024-09-22T03:37:52.688315+0000","last_active":"2024-09-22T03:37:52.671951+0000","last_peered":"2024-09-22T03:37:52.671951+0000","last_clean":"2024-09-22T03:37:52.671951+0000","last_became_active":"2024-09-22T03:23:07.873816+0000","last_became_peered":"2024-09-22T03:23:07.873816+0000","last_unstale":"2024-09-22T03:37:52.671951+0000","last_undegraded":"2024-09-22T03:37:52.671951+0000","last_fullsized":"2024-09-22T03:37:52.671951+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:52.688103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:52.688103+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:52.234895+0000","last_change":"2024-09-22T03:37:52.246626+0000","last_active":"2024-09-22T03:37:52.234895+0000","last_peered":"2024-09-22T03:37:52.234895+0000","last_clean":"2024-09-22T03:37:52.234895+0000","last_became_active":"2024-09-22T03:23:07.870336+0000","last_became_peered":"2024-09-22T03:23:07.870336+0000","last_unstale":"2024-09-22T03:37:52.234895+0000","last_undegraded":"2024-09-22T03:37:52.234895+0000","last_fullsized":"2024-09-22T03:37:52.234895+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:52.246508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:52.246508+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":327,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-22T03:37:51.798214+0000","last_change":"2024-09-22T03:37:51.813028+0000","last_active":"2024-09-22T03:37:51.798214+0000","last_peered":"2024-09-22T03:37:51.798214+0000","last_clean":"2024-09-22T03:37:51.798214+0000","last_became_active":"2024-09-22T03:23:07.884642+0000","last_became_peered":"2024-09-22T03:23:07.884642+0000","last_unstale":"2024-09-22T03:37:51.798214+0000","last_undegraded":"2024-09-22T03:37:51.798214+0000","last_fullsized":"2024-09-22T03:37:51.798214+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:37:51.812934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:37:51.812934+0000","last_clean_scrub_stamp":"2024-09-22T03:23:06.804798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214203592704,"internally_reserved":0,"allocated":7901184,"data_stored":7901184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1884,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295844,"statfs":{"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2228,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295500,"statfs":{"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1324,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296404,"statfs":{"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":2280,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52295448,"statfs":{"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53550538752,"internally_reserved":0,"allocated":2334720,"data_stored":2334720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551517696,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53550592000,"internally_reserved":0,"allocated":2281472,"data_stored":2281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53550944256,"internally_reserved":0,"allocated":1929216,"data_stored":1929216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:38:13.289 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-22T03:38:13.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:38:13.385+0000 7f1e558d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:38:13.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-22T03:38:13.395+0000 7f1e558d7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:38:13.588 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-22T03:38:13.588 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-22T03:38:13.589 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T03:38:13.589 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:38:18.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-22T03:38:19.691 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T03:38:19.691 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T03:38:19.691 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:38:23.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-22T03:38:23.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-22T03:38:25.793 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T03:38:25.793 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T03:38:25.793 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:38:29.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-22T03:38:29.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-22T03:38:31.895 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T03:38:31.895 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T03:38:31.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:38:34.244 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-22T03:38:34.244 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-22T03:38:37.997 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T03:38:37.997 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-22T03:38:37.998 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-22T03:38:37.998 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:38:38.031 INFO:tasks.ceph.mgr.x:Stopped 2024-09-22T03:38:38.031 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-22T03:38:38.031 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-22T03:38:38.031 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T03:38:38.046 INFO:tasks.ceph.mon.a:Stopped 2024-09-22T03:38:38.046 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-22T03:38:38.047 DEBUG:teuthology.orchestra.run.smithi129:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-22T03:38:38.125 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-22T03:38:38.125 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-22T03:38:38.274 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-22T03:38:38.274 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-22T03:38:38.340 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-22T03:38:38.340 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-22T03:38:38.435 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-22T03:38:38.435 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-22T03:38:38.502 INFO:tasks.ceph:Archiving mon data... 2024-09-22T03:38:38.504 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757/data/mon.a.tgz 2024-09-22T03:38:38.504 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-22T03:38:38.523 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.SaFbk7C7i7 2024-09-22T03:38:38.523 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SaFbk7C7i7 2024-09-22T03:38:38.649 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.SaFbk7C7i7 2024-09-22T03:38:38.735 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.SaFbk7C7i7 is 234KB 2024-09-22T03:38:38.800 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.SaFbk7C7i7 2024-09-22T03:38:38.816 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-22T03:38:38.816 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T03:38:38.857 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T03:38:38.935 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T03:38:38.953 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T03:38:38.953 INFO:tasks.ceph:Archiving crash dumps... 2024-09-22T03:38:38.955 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757/remote/smithi129/crash 2024-09-22T03:38:38.955 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T03:38:39.001 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757/remote/smithi144/crash 2024-09-22T03:38:39.001 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T03:38:39.043 INFO:tasks.ceph:Compressing logs... 2024-09-22T03:38:39.044 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T03:38:39.046 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T03:38:39.067 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-22T03:38:39.125 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27731.log 2024-09-22T03:38:39.125 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27651.log 2024-09-22T03:38:39.125 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-22T03:38:39.125 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27731.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27731.log.gz 2024-09-22T03:38:39.125 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27651.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-22T03:38:39.125 INFO:teuthology.orchestra.run.smithi144.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27651.log.gz 2024-09-22T03:38:39.125 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27691.log 2024-09-22T03:38:39.126 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-22T03:38:39.126 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27600.log 2024-09-22T03:38:39.126 INFO:teuthology.orchestra.run.smithi144.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27691.log.gz 2024-09-22T03:38:39.126 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.27600.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.27600.log.gz 2024-09-22T03:38:39.243 INFO:teuthology.orchestra.run.smithi144.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-22T03:38:39.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-22T03:38:39.292 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54058.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52128.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.54058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55212.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54058.log.gz 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28965.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52128.log.gz 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41341.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55212.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55212.log.gz 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32978.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28965.log.gz 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44466.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41341.log.gz 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64364.log 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32978.log.gz 2024-09-22T03:38:39.293 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56810.log 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44466.log.gz 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63401.log 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.64364.log.gz 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39979.log 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56810.log.gz 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39449.log 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63401.log.gz 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35002.log 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39979.log.gz 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35309.log 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39449.log.gz 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34141.log 2024-09-22T03:38:39.294 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35002.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59701.log 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35309.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35533.log 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34141.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57993.log 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59701.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64495.log 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35533.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34825.log 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57993.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36415.log 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64495.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46417.log 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34825.log.gz 2024-09-22T03:38:39.295 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31288.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36415.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62409.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46417.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58260.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31288.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47073.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62409.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61161.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58260.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47073.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47073.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59821.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48976.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61161.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64057.log 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59821.log.gz 2024-09-22T03:38:39.296 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60317.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48976.log.gz 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48885.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64057.log.gz 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37391.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60317.log.gz 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43438.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48885.log.gz 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31119.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37391.log.gz 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31684.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43438.log.gz 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43080.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31119.log.gz 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31070.log 2024-09-22T03:38:39.297 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31684.log.gz 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41564.log 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43080.log.gz 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57586.log 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31070.log.gz 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33608.log 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41564.log.gz 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64315.log 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57586.log.gz 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28654.log 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33608.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33608.log.gz 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56692.log 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64315.log.gz 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61638.log 2024-09-22T03:38:39.298 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28654.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39325.log 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56692.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54812.log 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61638.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63630.log 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39325.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29486.log 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54812.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37560.log 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63630.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49104.log 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29486.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60228.log 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37560.log.gz 2024-09-22T03:38:39.299 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53567.log 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49104.log.gz 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64106.log 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60228.log.gz 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37351.log 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53567.log.gz 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58885.log 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64106.log.gz 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36146.log 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37351.log.gz 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33963.log 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58885.log.gz 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36146.log.gz 2024-09-22T03:38:39.300 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52048.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33963.log.gz 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59870.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63928.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52048.log.gz 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44375.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59870.log.gz 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62588.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63928.log.gz 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45590.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44375.log.gz 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40426.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.62588.log.gz 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50415.log 2024-09-22T03:38:39.301 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45590.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46726.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40426.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37701.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50415.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42086.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46726.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66006.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37701.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33258.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42086.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60108.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66006.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34606.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33258.log.gz 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46766.log 2024-09-22T03:38:39.302 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60108.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53848.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34606.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68098.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46766.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65390.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53848.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53458.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68098.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51051.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65390.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35788.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53458.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51051.log.gz 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57019.log 2024-09-22T03:38:39.303 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.35788.log.gz 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52753.log 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42495.log 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57019.log.gz 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55559.log 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52753.log.gz 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60765.log 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42495.log.gz 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51779.log 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55559.log.gz 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51231.log 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60765.log.gz 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43298.log 2024-09-22T03:38:39.304 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51779.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46110.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51231.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42802.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43298.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43298.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48271.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46110.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44096.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42802.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43718.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48271.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57099.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44096.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64629.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43718.log.gz 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64994.log 2024-09-22T03:38:39.305 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57099.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64629.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64629.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67621.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64994.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64994.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50115.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr: 48.8% -- replaced with /var/log/ceph/ceph-client.admin.67621.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51919.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66832.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50115.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42544.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51919.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58340.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66832.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37511.log 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42544.log.gz 2024-09-22T03:38:39.306 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38676.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58340.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37511.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37511.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61281.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42593.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38676.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61281.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61281.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50504.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64546.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42593.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50504.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50504.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49500.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38854.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64546.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49500.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49500.log.gz 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55519.log 2024-09-22T03:38:39.307 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32221.log 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38854.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38854.log.gz 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33347.log 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55519.log.gz 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48191.log 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32221.log.gz 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33347.log: gzip -5 --verbose 49.6% -- -- replaced with /var/log/ceph/ceph-client.admin.33347.log.gz /var/log/ceph/ceph-client.admin.63710.log 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30880.log 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48191.log.gz 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63710.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.63710.log.gz 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38987.log 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68099.log 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30880.log.gz 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38987.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.38987.log.gz 2024-09-22T03:38:39.308 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44718.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39629.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68099.log.gz 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44718.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.59116.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44718.log.gz 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48578.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39629.log.gz 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44162.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59116.log.gz 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48578.log.gz 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48796.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44162.log.gz 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37750.log 2024-09-22T03:38:39.309 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30110.log 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.48796.log.gz 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54895.log 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37750.log.gz 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46279.log 50.0% 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30110.log.gz 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69257.log 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54895.log.gz 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46279.log: gzip -5 49.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.46279.log.gz -- 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.31595.log 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.69257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67831.log 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.69257.log.gz 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37431.log 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31595.log.gz 2024-09-22T03:38:39.310 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50762.log 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.67831.log.gz 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28536.log 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37431.log.gz 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35091.log 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50762.log.gz 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50031.log 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28536.log.gz 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68545.log 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35091.log.gz 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50031.log.gz 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68545.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.67130.log 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68545.log.gz 2024-09-22T03:38:39.311 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45713.log 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30699.log 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67130.log.gz 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40734.log 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45713.log.gz 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66792.log 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30699.log.gz 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41181.log 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40734.log.gz 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48687.log 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66792.log.gz 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55946.log 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41181.log.gz 2024-09-22T03:38:39.312 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35051.log 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48687.log.gz 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29242.log 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55946.log.gz 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35051.log.gz 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40286.log 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29242.log.gz 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67228.log 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62668.log 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40286.log.gz 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29625.log 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67228.log.gz 2024-09-22T03:38:39.313 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62668.log: gzip -5 --verbose 50.0% -- -- replaced with /var/log/ceph/ceph-client.admin.62668.log.gz /var/log/ceph/ceph-client.admin.66313.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61499.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29625.log.gz 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62826.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66313.log.gz 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68629.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61499.log.gz 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.62826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63352.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30558.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68629.log.gz 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28561.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.63352.log.gz 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30558.log.gz 2024-09-22T03:38:39.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28561.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.28561.log.gz 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53658.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41828.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47571.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65252.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53658.log.gz 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33020.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41828.log.gz 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51730.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65252.log.gz 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64413.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33020.log.gz 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47936.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51730.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51730.log.gz 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66195.log 2024-09-22T03:38:39.315 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64413.log.gz 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47936.log: gzip -5 50.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47936.log.gz -- 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.61778.log 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44243.log 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66195.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66195.log.gz 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30649.log 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61778.log.gz 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62628.log 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44243.log.gz 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69007.log 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30649.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30649.log.gz 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62628.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62628.log.gz 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49192.log 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64588.log 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.69007.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.69007.log.gz 2024-09-22T03:38:39.316 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49192.log.gz 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52793.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64588.log.gz 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38407.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52793.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52793.log.gz 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61330.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39572.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38407.log.gz 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67931.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61330.log.gz 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.39572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40384.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67931.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67931.log.gz 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.31426.log 2024-09-22T03:38:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40384.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40384.log.gz 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40814.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43249.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31426.log.gz 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40814.log: gzip -5 --verbose -- 49.2% /var/log/ceph/ceph-client.admin.67579.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40814.log.gz 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67248.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.43249.log.gz 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63023.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67579.log.gz 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67248.log.gz 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43627.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63023.log.gz 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62318.log: gzip -5 --verbose -- 47.6% /var/log/ceph/ceph-client.admin.63670.log 2024-09-22T03:38:39.318 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43627.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43627.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57237.log 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63670.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63670.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40138.log 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57237.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57237.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61379.log 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40138.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40138.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.57495.log 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61379.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61379.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39937.log 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57495.log: gzip -5 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57495.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65837.log 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39937.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39937.log.gz 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59025.log 2024-09-22T03:38:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65837.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65837.log.gz 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61072.log 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59025.log: gzip 50.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.59025.log.gz --verbose 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.60576.log 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28480.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61072.log.gz 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60576.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60576.log.gz 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47024.log 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28480.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28480.log.gz 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31922.log 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47024.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47024.log.gz 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44202.log 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31922.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31922.log.gz 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60367.log 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44202.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34005.log 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44202.log.gz 2024-09-22T03:38:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60367.log.gz 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51499.log 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34005.log.gz 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30790.log 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51499.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51499.log.gz 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45186.log 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30790.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40774.log 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30790.log.gz 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45544.log 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45186.log.gz 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50176.log 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40774.log.gz 2024-09-22T03:38:39.321 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68101.log 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45544.log.gz 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46815.log/var/log/ceph/ceph-client.admin.50176.log: 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50176.log.gz 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68101.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68101.log.gz 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.56434.log 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46815.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46815.log.gz 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35868.log 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56434.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56434.log.gz 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41390.log 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60716.log 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35868.log.gz 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49709.log 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41390.log.gz 2024-09-22T03:38:39.322 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68965.log 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60716.log: /var/log/ceph/ceph-client.admin.49709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34230.log 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60716.log.gz 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49709.log.gz 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30285.log 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68965.log.gz 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47753.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34230.log.gz 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30285.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30285.log.gz 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50813.log 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47753.log: /var/log/ceph/ceph-client.admin.39798.log 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47753.log.gz 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50813.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50813.log.gz 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41649.log 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63888.log 49.6% 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 2024-09-22T03:38:39.323 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41649.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41649.log.gz 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63750.log 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58556.log 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63888.log.gz 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63750.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.45495.log 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63750.log.gz 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58556.log: gzip 50.4% -- replaced with /var/log/ceph/ceph-client.admin.58556.log.gz 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52931.log 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45495.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.45495.log.gz 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65301.log 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52931.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49241.log 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52931.log.gz 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69217.log 2024-09-22T03:38:39.324 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65301.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.65301.log.gz 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46459.log 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49241.log.gz 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.69217.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.69217.log.gz 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.60974.log 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46459.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46459.log.gz 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39718.log 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42355.log 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60974.log.gz 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39718.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.42404.log 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39718.log.gz 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42355.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42355.log.gz 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58300.log 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49898.log 2024-09-22T03:38:39.325 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42404.log: /var/log/ceph/ceph-client.admin.58300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42404.log.gz 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58300.log.gz 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.49898.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41301.log 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49898.log.gz 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45851.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47662.log 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41301.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41301.log.gz 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55777.log 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50366.log 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47662.log.gz 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55777.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55777.log.gz 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.55826.log 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50366.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50366.log.gz 2024-09-22T03:38:39.326 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60894.log 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55826.log: gzip -5 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55826.log.gz 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50073.log 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60894.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60894.log.gz 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54935.log 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66921.log 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50073.log.gz 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54935.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54935.log.gz 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.40517.log 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66921.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66921.log.gz 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36644.log 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40517.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40517.log.gz 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.57277.log 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36644.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36644.log.gz 2024-09-22T03:38:39.327 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29485.log 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57277.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60814.log 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57277.log.gz 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66486.log 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29485.log.gz 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60487.log 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60814.log.gz 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63799.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66486.log.gz 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60487.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57824.log 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60487.log.gz 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33654.log 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63799.log.gz 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53438.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57824.log.gz 2024-09-22T03:38:39.328 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33654.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33654.log.gz 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45891.log 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53438.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53438.log.gz 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.50464.log 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45891.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45891.log.gz 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42264.log 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50464.log: gzip -5 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50464.log.gz 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64235.log 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42264.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42264.log.gz 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55113.log 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53757.log 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64235.log.gz 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55113.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55113.log.gz 2024-09-22T03:38:39.329 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.36595.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53757.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53757.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45633.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49758.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45633.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.42756.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45633.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49758.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38190.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49758.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62478.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38190.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38190.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.37471.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62478.log: 48.8% -- replaced with /var/log/ceph/ceph-client.admin.62478.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62003.log 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37471.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-09-22T03:38:39.330 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.55599.log 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62003.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62003.log.gz 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34270.log 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55599.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55599.log.gz 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.37132.log 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34270.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34270.log.gz 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51681.log 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56809.log 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37132.log.gz 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51681.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51681.log.gz 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.40559.log 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56809.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47164.log 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56809.log.gz 2024-09-22T03:38:39.331 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59165.log 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.40559.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47164.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.51541.log 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47164.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59165.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59165.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60934.log 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51541.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51541.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59214.log 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60934.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38636.log 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54761.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59214.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38636.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38636.log.gz 2024-09-22T03:38:39.332 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39678.log 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54761.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54761.log.gz 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33467.log 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39678.log.gz 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52177.log 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59434.log 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33467.log: /var/log/ceph/ceph-client.admin.52177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34566.log 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33467.log.gz 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52177.log.gz 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30930.log 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59434.log: 49.6%/var/log/ceph/ceph-client.admin.34566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69091.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34566.log.gz 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59434.log.gz 2024-09-22T03:38:39.333 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30930.log: gzip 50.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.43578.log 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30930.log.gz 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.69091.log: 88.8%gzip -- replaced with /var/log/ceph/ceph.audit.log.gz -5 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30369.log 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.69091.log.gz 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60059.log 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43578.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43578.log.gz 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28795.log 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30369.log.gz 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32628.log 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60059.log.gz 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59454.log 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28795.log.gz 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61682.log 2024-09-22T03:38:39.334 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32628.log.gz 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58209.log 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59454.log.gz 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45226.log 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61682.log.gz 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64155.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58209.log.gz 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45226.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45226.log.gz 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44907.log 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64155.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64155.log.gz 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.41083.log 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44907.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44907.log.gz 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57544.log 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54543.log 50.0% 2024-09-22T03:38:39.335 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41083.log.gz 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59661.log 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57544.log.gz 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49063.log 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54543.log.gz 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59661.log: gzip -5 --verbose 50.4% -- replaced with /var/log/ceph/ceph-client.admin.59661.log.gz 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.57635.log 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49063.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49063.log.gz 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67319.log 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57635.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57635.log.gz 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.36504.log 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67319.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67319.log.gz 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68839.log 2024-09-22T03:38:39.336 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31208.log 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36504.log.gz 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.68839.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41479.log 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68839.log.gz 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31208.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.31208.log.gz -5 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46368.log 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44637.log 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41479.log.gz 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44677.log 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46368.log.gz 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44637.log: -- /var/log/ceph/ceph-client.admin.33694.log 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44637.log.gz 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44677.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44677.log.gz 2024-09-22T03:38:39.337 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35260.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33694.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33694.log.gz 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.51142.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35260.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35260.log.gz 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37799.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51142.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51142.log.gz 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35966.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37799.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37799.log.gz 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30839.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37959.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35966.log.gz 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51191.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30839.log.gz 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54285.log 2024-09-22T03:38:39.338 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37959.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53587.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51191.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54285.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54285.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52435.log 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31824.log 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53587.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52435.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.42991.log 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52435.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42215.log 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31824.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56035.log 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42991.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42215.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42215.log.gz 2024-09-22T03:38:39.339 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.47262.log 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56035.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56035.log.gz 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28796.log 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47451.log 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28796.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.52266.log 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28796.log.gz 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47451.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47451.log.gz 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37839.log 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68671.log 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.52266.log.gz 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37839.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-client.admin.49362.log 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37839.log.gz 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68671.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.68671.log.gz -5 2024-09-22T03:38:39.340 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43669.log 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49669.log 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49362.log.gz 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67410.log 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43669.log.gz 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32042.log 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49669.log.gz 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48449.log 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.67410.log.gz 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58647.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32042.log.gz 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48449.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48449.log.gz 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30460.log 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61023.log 2024-09-22T03:38:39.341 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58647.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30460.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30460.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.32390.log 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61023.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61023.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32714.log 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32390.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32390.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.58507.log 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32714.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32714.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29116.log 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.58507.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29116.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29116.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.42313.log 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49302.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-09-22T03:38:39.342 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51450.log 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42313.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42313.log.gz 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.43961.log 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51450.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51450.log.gz 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53022.log 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43961.log: -- /var/log/ceph/ceph-client.admin.56612.log 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43961.log.gz 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53022.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.53022.log.gz 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52326.log 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49450.log 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56612.log.gz 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52326.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52326.log.gz 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61588.log 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31475.log 2024-09-22T03:38:39.343 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.49450.log.gz 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61588.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.0.44316.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61588.log.gz 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31475.log: 50.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.31475.log.gz 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.0.44316.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53898.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50953.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64275.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53898.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.30979.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64275.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64275.log.gz 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49019.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5/var/log/ceph/ceph-client.admin.30979.log: --verbose -- /var/log/ceph/ceph-client.admin.36197.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30979.log.gz 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49019.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31775.log 2024-09-22T03:38:39.344 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49019.log.gz 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53071.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36197.log.gz 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.31775.log: -- /var/log/ceph/ceph-client.admin.54987.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31775.log.gz 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53071.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53071.log.gz 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43200.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54987.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37001.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54987.log.gz 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49135.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43200.log.gz 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54854.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37001.log.gz 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56394.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.49135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37919.log 2024-09-22T03:38:39.345 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54854.log: /var/log/ceph/ceph-client.admin.56394.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54854.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.56394.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53120.log 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64945.log 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37919.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37919.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53120.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53120.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29841.log 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35575.log 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29841.log: /var/log/ceph/ceph-client.admin.64945.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64945.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29841.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45048.log 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36847.log 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35575.log: /var/log/ceph/ceph-client.admin.45048.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35575.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45048.log.gz 2024-09-22T03:38:39.346 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49281.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36847.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36847.log.gz 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49281.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49281.log.gz 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63441.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67179.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63441.log: /var/log/ceph/ceph-client.admin.47886.log: 49.6% 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63441.log.gz 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54721.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58745.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67179.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67179.log.gz 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54721.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54721.log.gz 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60268.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54463.log 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60268.log: /var/log/ceph/ceph-client.admin.58745.log: 49.6% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60268.log.gz 2024-09-22T03:38:39.347 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58745.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29158.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61961.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54463.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54463.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29158.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29158.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61961.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61961.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68416.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29388.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68416.log: /var/log/ceph/ceph-client.admin.41908.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68416.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30243.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29388.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66135.log 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30243.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-09-22T03:38:39.348 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30243.log.gz 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47612.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66135.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39542.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66135.log.gz 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46130.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47612.log: /var/log/ceph/ceph-client.admin.39542.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47612.log.gz 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39542.log.gz 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67705.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56245.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.46130.log.gz 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52833.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56245.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67705.log.gz 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56245.log.gz 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68755.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-09-22T03:38:39.349 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52833.log.gz 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59612.log 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40117.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68755.log.gz 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37021.log 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35131.log 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59612.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59612.log.gz 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37021.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37021.log.gz 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36057.log 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46904.log 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36057.log: /var/log/ceph/ceph-client.admin.35131.log: 50.4% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36057.log.gz 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35131.log.gz 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2024-09-22T03:38:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62178.log 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46904.log.gz 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45097.log 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41016.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62178.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32132.log 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62178.log.gz 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57297.log 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45097.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.45097.log.gz 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32132.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34099.log 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32132.log.gz 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53708.log 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34099.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57297.log.gz 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34099.log.gz 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-09-22T03:38:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53708.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53708.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41957.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32936.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41957.log: /var/log/ceph/ceph-client.admin.39366.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41957.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32488.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32936.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50911.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32488.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32488.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55071.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28797.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50911.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50911.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55071.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55071.log.gz 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32803.log 2024-09-22T03:38:39.352 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.28797.log: /var/log/ceph/ceph-client.admin.49981.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28797.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67932.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49981.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32803.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49981.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39500.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28386.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67932.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46319.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28386.log: 49.2% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28386.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39500.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48667.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66621.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46319.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35615.log 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66621.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48667.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66621.log.gz 2024-09-22T03:38:39.353 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45673.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62932.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35615.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35615.log.gz 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45673.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45673.log.gz 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55728.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67933.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55728.log: /var/log/ceph/ceph-client.admin.62932.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55728.log.gz 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62932.log.gz 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37172.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40650.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67933.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67933.log.gz 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37172.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.30741.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30741.log: /var/log/ceph/ceph-client.admin.40650.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30741.log.gz 2024-09-22T03:38:39.354 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40650.log.gz 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48627.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46239.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46239.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.48627.log.gz 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46239.log.gz 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34526.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30059.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48062.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.34526.log.gz 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48529.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30059.log.gz/var/log/ceph/ceph-client.admin.48062.log: 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48062.log.gz 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63121.log 2024-09-22T03:38:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48529.log: -- /var/log/ceph/ceph-client.admin.43040.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48529.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54672.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43040.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63121.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37212.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39202.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54672.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54672.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37212.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37212.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54176.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64763.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54176.log: /var/log/ceph/ceph-client.admin.39202.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39202.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54176.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68417.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64763.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57139.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64763.log.gz 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57775.log 2024-09-22T03:38:39.356 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68417.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68417.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57139.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.57139.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67050.log 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66872.log 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57775.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57775.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67050.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67050.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39070.log 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56532.log 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66872.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66872.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39070.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39070.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42126.log 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56127.log 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56532.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56532.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42126.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51820.log 2024-09-22T03:38:39.357 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66970.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56127.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56127.log.gz 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51820.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32301.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51820.log.gz 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58033.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66970.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66970.log.gz 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32301.log: 50.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47803.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32301.log.gz 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31377.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58033.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58033.log.gz 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47803.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29921.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47803.log.gz 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55421.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31377.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31377.log.gz 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29921.log: 50.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34331.log 2024-09-22T03:38:39.358 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29921.log.gz 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52713.log 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55421.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55421.log.gz 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.34331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48082.log 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52713.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52713.log.gz 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29074.log 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48082.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48082.log.gz 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36959.log 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29074.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29074.log.gz 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32761.log 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36959.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36959.log.gz 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59354.log 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32761.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32761.log.gz 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29200.log 2024-09-22T03:38:39.359 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59354.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59354.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64721.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29200.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29200.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64721.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64721.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53393.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42453.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58576.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53393.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53393.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43487.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58576.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58576.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63968.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43487.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43487.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35442.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63968.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63968.log.gz 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62548.log 2024-09-22T03:38:39.360 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35442.log: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.35442.log.gz 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65608.log 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62548.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62548.log.gz 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58458.log 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65608.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65608.log.gz 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44767.log 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58458.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58458.log.gz 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68797.log 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44767.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44767.log.gz 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59305.log 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68797.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68797.log.gz 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36553.log 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59305.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59305.log.gz 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38231.log 2024-09-22T03:38:39.361 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36553.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36553.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62269.log 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38231.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38231.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68438.log 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62269.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62269.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68438.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68438.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31733.log 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39839.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60407.log 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31733.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31733.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60407.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60407.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40692.log 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52008.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-09-22T03:38:39.362 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64455.log 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40692.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40692.log.gz 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54365.log 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64455.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64455.log.gz 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33565.log 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54365.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54365.log.gz 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38926.log 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33565.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33565.log.gz 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32530.log 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38926.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38926.log.gz 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32530.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.32530.log.gz 2024-09-22T03:38:39.363 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53978.log 2024-09-22T03:38:39.365 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67663.log 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53978.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53978.log.gz 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66095.log 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67663.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67663.log.gz 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55866.log 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66095.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66095.log.gz 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40077.log 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55866.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55866.log.gz 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44417.log 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40077.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40077.log.gz 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51266.log 2024-09-22T03:38:39.366 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44417.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44417.log.gz 2024-09-22T03:38:39.372 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55679.log 2024-09-22T03:38:39.373 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51266.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51266.log.gz 2024-09-22T03:38:39.381 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47531.log 2024-09-22T03:38:39.382 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55679.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55679.log.gz 2024-09-22T03:38:39.398 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57059.log 2024-09-22T03:38:39.398 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47531.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47531.log.gz 2024-09-22T03:38:39.407 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65172.log 2024-09-22T03:38:39.417 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60010.log 2024-09-22T03:38:39.417 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65172.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65172.log.gz 2024-09-22T03:38:39.420 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.57059.log.gz 2024-09-22T03:38:39.425 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57188.log 2024-09-22T03:38:39.428 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60010.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60010.log.gz 2024-09-22T03:38:39.440 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38716.log 2024-09-22T03:38:39.441 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57188.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.57188.log.gz 2024-09-22T03:38:39.449 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52978.log 2024-09-22T03:38:39.459 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52533.log 2024-09-22T03:38:39.459 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52978.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52978.log.gz 2024-09-22T03:38:39.462 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz 2024-09-22T03:38:39.467 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44816.log 2024-09-22T03:38:39.480 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52533.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52533.log.gz 2024-09-22T03:38:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28966.log 2024-09-22T03:38:39.490 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44816.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44816.log.gz 2024-09-22T03:38:39.504 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57845.log 2024-09-22T03:38:39.505 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28966.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28966.log.gz 2024-09-22T03:38:39.513 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47122.log 2024-09-22T03:38:39.523 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58787.log 2024-09-22T03:38:39.523 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47122.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47122.log.gz 2024-09-22T03:38:39.526 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57845.log.gz 2024-09-22T03:38:39.531 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50722.log 2024-09-22T03:38:39.543 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47491.log 2024-09-22T03:38:39.543 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50722.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50722.log.gz 2024-09-22T03:38:39.544 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58787.log.gz 2024-09-22T03:38:39.551 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50624.log 2024-09-22T03:38:39.560 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32843.log 2024-09-22T03:38:39.564 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50624.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47491.log.gz 2024-09-22T03:38:39.569 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57446.log 2024-09-22T03:38:39.573 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32843.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.50624.log.gz 2024-09-22T03:38:39.578 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49807.log 2024-09-22T03:38:39.582 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57446.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32843.log.gz 2024-09-22T03:38:39.587 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63212.log 2024-09-22T03:38:39.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49807.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57446.log.gz 2024-09-22T03:38:39.596 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29284.log 2024-09-22T03:38:39.600 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63212.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49807.log.gz 2024-09-22T03:38:39.610 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63212.log.gz 2024-09-22T03:38:39.610 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60854.log 2024-09-22T03:38:39.610 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29284.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29284.log.gz 2024-09-22T03:38:39.626 INFO:teuthology.orchestra.run.smithi129.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.62826.log.gz 2024-09-22T03:38:39.626 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34764.log 2024-09-22T03:38:39.628 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60854.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60854.log.gz 2024-09-22T03:38:39.628 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-09-22T03:38:39.629 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34764.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34764.log.gz 2024-09-22T03:38:39.629 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29712.log 2024-09-22T03:38:39.629 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34913.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-09-22T03:38:39.631 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45146.log 2024-09-22T03:38:39.631 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29712.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29712.log.gz 2024-09-22T03:38:39.631 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67789.log 2024-09-22T03:38:39.631 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45146.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45146.log.gz 2024-09-22T03:38:39.633 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65350.log 2024-09-22T03:38:39.633 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67789.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67789.log.gz 2024-09-22T03:38:39.633 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-09-22T03:38:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65350.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65350.log.gz 2024-09-22T03:38:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61539.log 2024-09-22T03:38:39.635 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38805.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-09-22T03:38:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33427.log 2024-09-22T03:38:39.636 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61539.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61539.log.gz 2024-09-22T03:38:39.637 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42006.log 2024-09-22T03:38:39.638 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33427.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33427.log.gz 2024-09-22T03:38:39.638 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64903.log 2024-09-22T03:38:39.639 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42006.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42006.log.gz 2024-09-22T03:38:39.639 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46508.log 2024-09-22T03:38:39.639 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64903.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.64903.log.gz 2024-09-22T03:38:39.641 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28926.log 2024-09-22T03:38:39.641 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46508.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46508.log.gz 2024-09-22T03:38:39.641 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38057.log 2024-09-22T03:38:39.641 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28926.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28926.log.gz 2024-09-22T03:38:39.643 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33734.log 2024-09-22T03:38:39.643 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38057.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38057.log.gz 2024-09-22T03:38:39.643 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30193.log 2024-09-22T03:38:39.644 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33734.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33734.log.gz 2024-09-22T03:38:39.645 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36693.log 2024-09-22T03:38:39.645 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30193.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30193.log.gz 2024-09-22T03:38:39.646 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44295.log 2024-09-22T03:38:39.646 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36693.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36693.log.gz 2024-09-22T03:38:39.646 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35828.log 2024-09-22T03:38:39.648 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44295.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44295.log.gz 2024-09-22T03:38:39.648 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-09-22T03:38:39.648 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35828.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35828.log.gz 2024-09-22T03:38:39.649 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56652.log 2024-09-22T03:38:39.649 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39140.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-09-22T03:38:39.650 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67930.log 2024-09-22T03:38:39.650 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56652.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56652.log.gz 2024-09-22T03:38:39.652 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38756.log 2024-09-22T03:38:39.652 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67930.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67930.log.gz 2024-09-22T03:38:39.652 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54503.log 2024-09-22T03:38:39.652 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38756.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38756.log.gz 2024-09-22T03:38:39.654 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50673.log 2024-09-22T03:38:39.654 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54503.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54503.log.gz 2024-09-22T03:38:39.654 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57953.log 2024-09-22T03:38:39.655 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50673.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50673.log.gz 2024-09-22T03:38:39.655 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60148.log 2024-09-22T03:38:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57953.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57953.log.gz 2024-09-22T03:38:39.656 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65470.log 2024-09-22T03:38:39.657 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60148.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60148.log.gz 2024-09-22T03:38:39.657 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60447.log 2024-09-22T03:38:39.659 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65470.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65470.log.gz 2024-09-22T03:38:39.659 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49940.log 2024-09-22T03:38:39.659 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60447.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60447.log.gz 2024-09-22T03:38:39.660 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50286.log 2024-09-22T03:38:39.661 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49940.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49940.log.gz 2024-09-22T03:38:39.661 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56930.log 2024-09-22T03:38:39.662 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50286.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50286.log.gz 2024-09-22T03:38:39.662 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34953.log 2024-09-22T03:38:39.662 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56930.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56930.log.gz 2024-09-22T03:38:39.664 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43347.log 2024-09-22T03:38:39.664 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34953.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34953.log.gz 2024-09-22T03:38:39.664 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65649.log 2024-09-22T03:38:39.664 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43347.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43347.log.gz 2024-09-22T03:38:39.666 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66537.log 2024-09-22T03:38:39.666 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65649.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65649.log.gz 2024-09-22T03:38:39.667 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51400.log 2024-09-22T03:38:39.667 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66537.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66537.log.gz 2024-09-22T03:38:39.669 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56881.log 2024-09-22T03:38:39.669 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51400.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51400.log.gz 2024-09-22T03:38:39.670 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32082.log 2024-09-22T03:38:39.670 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56881.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56881.log.gz 2024-09-22T03:38:39.670 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66641.log 2024-09-22T03:38:39.672 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32082.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32082.log.gz 2024-09-22T03:38:39.673 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61827.log 2024-09-22T03:38:39.673 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66641.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66641.log.gz 2024-09-22T03:38:39.673 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34646.log 2024-09-22T03:38:39.673 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61827.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61827.log.gz 2024-09-22T03:38:39.673 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33118.log 2024-09-22T03:38:39.675 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34646.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34646.log.gz 2024-09-22T03:38:39.675 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54018.log 2024-09-22T03:38:39.675 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33118.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33118.log.gz 2024-09-22T03:38:39.675 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59256.log 2024-09-22T03:38:39.675 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54018.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54018.log.gz 2024-09-22T03:38:39.678 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40028.log 2024-09-22T03:38:39.678 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59256.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59256.log.gz 2024-09-22T03:38:39.678 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41748.log 2024-09-22T03:38:39.678 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40028.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40028.log.gz 2024-09-22T03:38:39.678 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59563.log 2024-09-22T03:38:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41748.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41748.log.gz 2024-09-22T03:38:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56085.log 2024-09-22T03:38:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59563.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59563.log.gz 2024-09-22T03:38:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28230.log 2024-09-22T03:38:39.682 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56085.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56085.log.gz 2024-09-22T03:38:39.682 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49410.log 2024-09-22T03:38:39.682 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28230.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28230.log.gz 2024-09-22T03:38:39.684 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68100.log 2024-09-22T03:38:39.684 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49410.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49410.log.gz 2024-09-22T03:38:39.684 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51968.log 2024-09-22T03:38:39.684 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68100.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68100.log.gz 2024-09-22T03:38:39.686 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31635.log 2024-09-22T03:38:39.686 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51968.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51968.log.gz 2024-09-22T03:38:39.686 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65510.log 2024-09-22T03:38:39.686 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31635.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31635.log.gz 2024-09-22T03:38:39.688 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31168.log 2024-09-22T03:38:39.688 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65510.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65510.log.gz 2024-09-22T03:38:39.688 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44503.log 2024-09-22T03:38:39.689 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31168.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31168.log.gz 2024-09-22T03:38:39.690 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36784.log 2024-09-22T03:38:39.691 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44503.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.44503.log.gz 2024-09-22T03:38:39.691 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67747.log 2024-09-22T03:38:39.691 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36784.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36784.log.gz 2024-09-22T03:38:39.697 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66264.log 2024-09-22T03:38:39.697 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67747.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.67747.log.gz 2024-09-22T03:38:39.697 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64805.log 2024-09-22T03:38:39.697 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66264.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66264.log.gz 2024-09-22T03:38:39.697 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33209.log 2024-09-22T03:38:39.697 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64805.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.64805.log.gz 2024-09-22T03:38:39.698 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56176.log 2024-09-22T03:38:39.698 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33209.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33209.log.gz 2024-09-22T03:38:39.698 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65559.log 2024-09-22T03:38:39.698 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56176.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56176.log.gz 2024-09-22T03:38:39.698 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39245.log 2024-09-22T03:38:39.698 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65559.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65559.log.gz 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52395.log 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39245.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39245.log.gz 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38547.log 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52395.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36917.log 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38547.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38547.log.gz 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67368.log 2024-09-22T03:38:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36917.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36917.log.gz 2024-09-22T03:38:39.705 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35659.log 2024-09-22T03:38:39.705 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67368.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67368.log.gz 2024-09-22T03:38:39.705 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62974.log 2024-09-22T03:38:39.705 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35659.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35659.log.gz 2024-09-22T03:38:39.705 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33307.log 2024-09-22T03:38:39.705 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62974.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62974.log.gz 2024-09-22T03:38:39.708 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61459.log 2024-09-22T03:38:39.708 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33307.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33307.log.gz 2024-09-22T03:38:39.708 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29326.log 2024-09-22T03:38:39.708 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61459.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61459.log.gz 2024-09-22T03:38:39.708 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41221.log 2024-09-22T03:38:39.708 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29326.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29326.log.gz 2024-09-22T03:38:39.710 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58082.log 2024-09-22T03:38:39.710 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41221.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41221.log.gz 2024-09-22T03:38:39.710 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32181.log 2024-09-22T03:38:39.710 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58082.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58082.log.gz 2024-09-22T03:38:39.713 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40894.log 2024-09-22T03:38:39.714 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32181.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.32181.log.gz 2024-09-22T03:38:39.714 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60527.log 2024-09-22T03:38:39.714 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40894.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40894.log.gz 2024-09-22T03:38:39.714 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48020.log 2024-09-22T03:38:39.714 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60527.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60527.log.gz 2024-09-22T03:38:39.716 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-09-22T03:38:39.716 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48020.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48020.log.gz 2024-09-22T03:38:39.717 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28696.log 2024-09-22T03:38:39.717 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57905.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-09-22T03:38:39.717 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52088.log 2024-09-22T03:38:39.719 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28696.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28696.log.gz 2024-09-22T03:38:39.719 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-09-22T03:38:39.719 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52088.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52088.log.gz 2024-09-22T03:38:39.719 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38008.log 2024-09-22T03:38:39.719 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35462.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-09-22T03:38:39.721 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42046.log 2024-09-22T03:38:39.721 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38008.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38008.log.gz 2024-09-22T03:38:39.721 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2024-09-22T03:38:39.721 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42046.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42046.log.gz 2024-09-22T03:38:39.723 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49847.log 2024-09-22T03:38:39.723 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43858.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2024-09-22T03:38:39.724 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34388.log 2024-09-22T03:38:39.724 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49847.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49847.log.gz 2024-09-22T03:38:39.725 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31028.log 2024-09-22T03:38:39.726 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34388.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34388.log.gz 2024-09-22T03:38:39.726 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43160.log 2024-09-22T03:38:39.726 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31028.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31028.log.gz 2024-09-22T03:38:39.728 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62752.log 2024-09-22T03:38:39.728 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43160.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43160.log.gz 2024-09-22T03:38:39.729 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53938.log 2024-09-22T03:38:39.729 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62752.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62752.log.gz 2024-09-22T03:38:39.729 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55381.log 2024-09-22T03:38:39.731 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53938.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53938.log.gz 2024-09-22T03:38:39.731 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58696.log 2024-09-22T03:38:39.731 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55381.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55381.log.gz 2024-09-22T03:38:39.731 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41520.log 2024-09-22T03:38:39.731 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58696.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58696.log.gz 2024-09-22T03:38:39.733 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-09-22T03:38:39.733 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41520.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41520.log.gz 2024-09-22T03:38:39.733 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54156.log 2024-09-22T03:38:39.734 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34744.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-09-22T03:38:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50177.log 2024-09-22T03:38:39.736 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54156.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54156.log.gz 2024-09-22T03:38:39.736 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33069.log 2024-09-22T03:38:39.736 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50177.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50177.log.gz 2024-09-22T03:38:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63303.log 2024-09-22T03:38:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33069.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33069.log.gz 2024-09-22T03:38:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64195.log 2024-09-22T03:38:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63303.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63303.log.gz 2024-09-22T03:38:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58976.log 2024-09-22T03:38:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64195.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64195.log.gz 2024-09-22T03:38:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37252.log 2024-09-22T03:38:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58976.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58976.log.gz 2024-09-22T03:38:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40206.log 2024-09-22T03:38:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37252.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37252.log.gz 2024-09-22T03:38:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57684.log 2024-09-22T03:38:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40206.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40206.log.gz 2024-09-22T03:38:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-09-22T03:38:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57684.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.57684.log.gz 2024-09-22T03:38:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63163.log 2024-09-22T03:38:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46637.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-09-22T03:38:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61201.log 2024-09-22T03:38:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63163.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63163.log.gz 2024-09-22T03:38:39.747 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48369.log 2024-09-22T03:38:39.748 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61201.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61201.log.gz 2024-09-22T03:38:39.748 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66362.log 2024-09-22T03:38:39.748 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48369.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48369.log.gz 2024-09-22T03:38:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28414.log 2024-09-22T03:38:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66362.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.66362.log.gz 2024-09-22T03:38:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29487.log 2024-09-22T03:38:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28414.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28414.log.gz 2024-09-22T03:38:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52226.log 2024-09-22T03:38:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29487.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29487.log.gz 2024-09-22T03:38:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35917.log 2024-09-22T03:38:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52226.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52226.log.gz 2024-09-22T03:38:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68268.log 2024-09-22T03:38:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35917.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35917.log.gz 2024-09-22T03:38:39.756 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43767.log 2024-09-22T03:38:39.756 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68268.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68268.log.gz 2024-09-22T03:38:39.756 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35171.log 2024-09-22T03:38:39.756 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43767.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43767.log.gz 2024-09-22T03:38:39.756 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56979.log 2024-09-22T03:38:39.757 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35171.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35171.log.gz 2024-09-22T03:38:39.759 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42942.log 2024-09-22T03:38:39.759 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56979.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56979.log.gz 2024-09-22T03:38:39.759 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29881.log 2024-09-22T03:38:39.759 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42942.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42942.log.gz 2024-09-22T03:38:39.759 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59074.log 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29881.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29881.log.gz 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58380.log 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59074.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59074.log.gz 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.39572.log.gz 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53211.log 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5/var/log/ceph/ceph-client.admin.58380.log: --verbose -- /var/log/ceph/ceph-client.admin.45753.log 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58380.log.gz 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53211.log: /var/log/ceph/ceph-client.admin.45753.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53211.log.gz 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45753.log.gz 2024-09-22T03:38:39.763 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52673.log 2024-09-22T03:38:39.765 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63590.log 2024-09-22T03:38:39.765 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.52673.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52673.log.gz 2024-09-22T03:38:39.765 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63261.log 2024-09-22T03:38:39.765 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63590.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63590.log.gz 2024-09-22T03:38:39.765 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57726.log 2024-09-22T03:38:39.766 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63261.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63261.log.gz 2024-09-22T03:38:39.766 INFO:teuthology.orchestra.run.smithi129.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-22T03:38:39.766 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35211.log 2024-09-22T03:38:39.766 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41608.log 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57726.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.57726.log.gz 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35211.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35211.log.gz 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54414.log 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34161.log 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54414.log: /var/log/ceph/ceph-client.admin.41608.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41608.log.gz 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54414.log.gz 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51093.log 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.34161.log: -- /var/log/ceph/ceph-client.admin.46944.log 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34161.log.gz 2024-09-22T03:38:39.852 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62709.log 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46944.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51093.log.gz 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46944.log.gz 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33167.log 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55261.log 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62709.log.gz 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45931.log 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55261.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33167.log.gz 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55261.log.gz 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36106.log 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43899.log 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45931.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45931.log.gz 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36106.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36106.log.gz 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46657.log 2024-09-22T03:38:39.853 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45802.log 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr: 94.1%/var/log/ceph/ceph-client.admin.46657.log: /var/log/ceph/ceph-client.admin.43899.log: -- replaced with /var/log/ceph/ceph-client.0.49135.log.gz 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46657.log.gz 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43899.log.gz 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66712.log 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50862.log 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45802.log.gz 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37609.log 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66712.log.gz 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38148.log 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50862.log: /var/log/ceph/ceph-client.admin.37609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58927.log 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50862.log.gz 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37609.log.gz 2024-09-22T03:38:39.854 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65043.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38148.log: /var/log/ceph/ceph-client.admin.58927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38148.log.gz 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58927.log.gz 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65043.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65043.log.gz 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29761.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48231.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.32439.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29761.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29761.log.gz 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60188.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32439.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.32439.log.gz 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45266.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37302.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60188.log: /var/log/ceph/ceph-client.admin.45266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68587.log 2024-09-22T03:38:39.855 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45266.log.gz 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60188.log.gz 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37302.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37302.log.gz 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30600.log 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68587.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68587.log.gz 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59919.log 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.30600.log: /var/log/ceph/ceph-client.admin.61910.log 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30600.log.gz 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59919.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59919.log.gz 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50544.log 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52882.log 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61910.log: /var/log/ceph/ceph-client.admin.50544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53162.log 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61910.log.gz 2024-09-22T03:38:39.856 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50544.log.gz 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28612.log 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52882.log: /var/log/ceph/ceph-client.admin.53162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37092.log 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52882.log.gz 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53162.log.gz 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28612.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28612.log.gz 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53302.log 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63492.log 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37092.log: /var/log/ceph/ceph-client.admin.53302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56572.log 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37092.log.gz 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53302.log.gz 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33823.log 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63492.log: /var/log/ceph/ceph-client.admin.56572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51870.log 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63492.log.gz -- replaced with /var/log/ceph/ceph-client.admin.56572.log.gz 2024-09-22T03:38:39.857 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33823.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33823.log.gz 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68377.log 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51870.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.51870.log.gz 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68377.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55341.log 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68377.log.gz 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59741.log 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65132.log 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55341.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55341.log.gz 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60674.log 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59741.log: /var/log/ceph/ceph-client.admin.65132.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59741.log.gz -5 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65748.log 2024-09-22T03:38:39.858 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65132.log.gz 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31555.log 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60674.log: /var/log/ceph/ceph-client.admin.65748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51002.log 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.60674.log.gz 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65748.log.gz 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31555.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31555.log.gz 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46864.log 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43816.log 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51002.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51002.log.gz 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43962.log 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46864.log: gzip/var/log/ceph/ceph-client.admin.43816.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34695.log 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43816.log.gz 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46864.log.gz/var/log/ceph/ceph-client.admin.43962.log: 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43962.log.gz 2024-09-22T03:38:39.859 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31873.log 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63541.log 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31248.log 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31873.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34695.log.gz 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31873.log.gz 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63541.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.63541.log.gz 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38447.log 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31248.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31248.log.gz 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29970.log 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38447.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38447.log.gz 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.29970.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33914.log 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29970.log.gz 2024-09-22T03:38:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29801.log 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62136.log 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33914.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33914.log.gz 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66579.log 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29801.log: /var/log/ceph/ceph-client.admin.62136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28925.log 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62136.log.gz 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29801.log.gz 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66579.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66579.log.gz 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67010.log 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54107.log 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28925.log: /var/log/ceph/ceph-client.admin.67010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32261.log 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28925.log.gz 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67010.log.gz 2024-09-22T03:38:39.861 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54107.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.54623.log 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54107.log.gz 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32261.log: /var/log/ceph/ceph-client.admin.54623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59959.log 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32261.log.gz 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37650.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44957.log 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64678.log 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42633.log 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44957.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.59959.log.gz 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44957.log.gz 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64678.log: gzip -5 49.2% -- replaced with /var/log/ceph/ceph-client.admin.64678.log.gz 2024-09-22T03:38:39.862 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39285.log 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42633.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42633.log.gz 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61870.log 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39285.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39285.log.gz 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55470.log 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65430.log 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36246.log 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55470.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61870.log.gz 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.55470.log.gz 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65430.log: gzip -5 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65430.log.gz 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68269.log 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36246.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55162.log 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36246.log.gz 2024-09-22T03:38:39.863 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46597.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68269.log.gz 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50584.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55162.log: /var/log/ceph/ceph-client.admin.46597.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68713.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55162.log.gz 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46597.log.gz 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.50584.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50584.log.gz 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.66055.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68713.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68713.log.gz 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43982.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69049.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66055.log: /var/log/ceph/ceph-client.admin.43982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41132.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66055.log.gz 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43982.log.gz 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.69049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35748.log 2024-09-22T03:38:39.864 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.69049.log.gz 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47307.log 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41132.log.gz 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35748.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35748.log.gz 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55301.log 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69133.log 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47307.log: /var/log/ceph/ceph-client.admin.55301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65087.log 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47307.log.gz 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55301.log.gz 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.69133.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48752.log 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69133.log.gz 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41788.log 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65087.log.gz 2024-09-22T03:38:39.865 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.48752.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62458.log 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48752.log.gz 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr: 94.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67537.log 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.0.44316.log.gz 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41788.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41788.log.gz 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43389.log 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62458.log.gz 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47402.log 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67537.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67537.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51312.log 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43389.log: /var/log/ceph/ceph-client.admin.47402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36335.log 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47402.log.gz 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43389.log.gz 2024-09-22T03:38:39.866 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51312.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51312.log.gz 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.65797.log 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36335.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36335.log.gz 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36455.log 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65797.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32894.log 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65797.log.gz 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39120.log 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36455.log.gz 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39029.log 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32894.log.gz 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5/var/log/ceph/ceph-client.admin.39120.log: --verbose -- /var/log/ceph/ceph.tmp-client.admin.27348.log 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39120.log.gz 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41261.log 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39029.log.gz 2024-09-22T03:38:39.867 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.tmp-client.admin.27348.log: gzip -5 --verbose -- 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27348.log.gz 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.28798.log 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41261.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41261.log.gz 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62883.log 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28798.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28798.log.gz 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59394.log 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62883.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.34045.log 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62883.log.gz 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59394.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59394.log.gz 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34045.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34045.log.gz 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62220.log 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48836.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32341.log 2024-09-22T03:38:39.868 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62220.log: 50.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42851.log 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62220.log.gz 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59523.log 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32341.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.32341.log.gz 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52624.log 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42851.log.gz 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59523.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59523.log.gz 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34437.log 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55986.log 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52624.log.gz 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63988.log 49.2% 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34437.log.gz 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65877.log 2024-09-22T03:38:39.869 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55986.log.gz 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40246.log 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63988.log.gz 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68881.log 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65877.log.gz 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58169.log 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40246.log.gz 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35329.log 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.68881.log.gz 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58124.log 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.58169.log.gz 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67090.log 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35329.log: /var/log/ceph/ceph-client.admin.58124.log: gzip 50.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.35329.log.gz --verbose 2024-09-22T03:38:39.870 INFO:teuthology.orchestra.run.smithi129.stderr: -- /var/log/ceph/ceph-client.admin.56483.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58124.log.gz 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36713.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67090.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67090.log.gz 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53510.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56483.log.gz 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.36713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31515.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36713.log.gz 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.53510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68267.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40600.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31515.log.gz 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53799.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68267.log.gz 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54233.log 2024-09-22T03:38:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40600.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40600.log.gz 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62043.log 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53799.log: /var/log/ceph/ceph-client.admin.54233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66404.log 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.53799.log.gz 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54233.log.gz 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62043.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62043.log.gz 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48489.log 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60625.log 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66404.log: /var/log/ceph/ceph-client.admin.48489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30152.log 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66404.log.gz 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48489.log.gz 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.60625.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.60625.log.gz 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42147.log 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55639.log 2024-09-22T03:38:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30152.log: /var/log/ceph/ceph-client.admin.42147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39758.log 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30152.log.gz 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42147.log.gz 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33516.log 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55639.log: /var/log/ceph/ceph-client.admin.39758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32579.log 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55639.log.gz 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39758.log.gz 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33516.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.33516.log.gz 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42900.log 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32579.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32579.log.gz 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38592.log 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42900.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42900.log.gz 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63072.log 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38592.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38592.log.gz 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44997.log 2024-09-22T03:38:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29655.log 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63072.log: /var/log/ceph/ceph-client.admin.44997.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63072.log.gz 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38896.log 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44997.log.gz 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68923.log 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.29655.log: /var/log/ceph/ceph-client.admin.38896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62796.log 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38896.log.gz 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35400.log 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68923.log.gz/var/log/ceph/ceph-client.admin.62796.log: 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62796.log.gz 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38323.log 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35400.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35400.log.gz 2024-09-22T03:38:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40992.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65957.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61419.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38323.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38323.log.gz 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65957.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65957.log.gz 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56741.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61241.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56741.log: /var/log/ceph/ceph-client.admin.61419.log: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56741.log.gz 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61419.log.gz 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31962.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54325.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.61241.log.gz 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45355.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31962.log.gz 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32669.log 2024-09-22T03:38:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45355.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.54325.log.gz 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45355.log.gz 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52484.log 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.32669.log: -- /var/log/ceph/ceph-client.admin.36295.log 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32669.log.gz 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61729.log 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0%/var/log/ceph/ceph-client.admin.36295.log: -- replaced with /var/log/ceph/ceph-client.admin.52484.log.gz 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36295.log.gz 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51360.log 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33774.log 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61729.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61729.log.gz 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51360.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51360.log.gz 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62094.log 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36008.log 2024-09-22T03:38:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33774.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33774.log.gz 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62094.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.62094.log.gz 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47978.log 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44597.log 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47978.log: /var/log/ceph/ceph-client.admin.36008.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36008.log.gz 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47978.log.gz 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56225.log 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33387.log 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44597.log: /var/log/ceph/ceph-client.admin.56225.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44597.log.gz 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.56225.log.gz 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47845.log 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58401.log 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33387.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33387.log.gz 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47845.log: 50.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38098.log 2024-09-22T03:38:39.877 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47845.log.gz 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42673.log 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.58401.log: /var/log/ceph/ceph-client.admin.38098.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38098.log.gz 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34486.log 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42673.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42673.log.gz 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38498.log 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34486.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34486.log.gz 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52575.log 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38498.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38498.log.gz 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59781.log 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52575.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.52575.log.gz 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34310.log 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.59781.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.59781.log.gz 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46557.log 2024-09-22T03:38:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34310.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34310.log.gz 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41439.log 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46557.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46557.log.gz 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30327.log 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41439.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41439.log.gz 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45306.log 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30327.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30327.log.gz 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46984.log 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45306.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45306.log.gz 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30389.log 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46984.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46984.log.gz 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48409.log 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30389.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30389.log.gz 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56308.log 2024-09-22T03:38:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48409.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48409.log.gz 2024-09-22T03:38:39.880 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28151.log 2024-09-22T03:38:39.897 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56308.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56308.log.gz 2024-09-22T03:38:39.897 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35708.log 2024-09-22T03:38:39.897 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.28151.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28151.log.gz 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32002.log 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35708.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35708.log.gz 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40335.log 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.32002.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32002.log.gz 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38365.log 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40335.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40335.log.gz 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52306.log 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38365.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38365.log.gz 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56354.log 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.52306.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.52306.log.gz 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40854.log 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.56354.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.56354.log.gz 2024-09-22T03:38:39.898 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40943.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40854.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40854.log.gz 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57405.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40943.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40943.log.gz 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65699.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57405.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.57405.log.gz 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68209.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65699.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65699.log.gz 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29583.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.68209.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.68209.log.gz 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47712.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.29583.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29583.log.gz 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61117.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47712.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47712.log.gz 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40467.log 2024-09-22T03:38:39.899 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.61117.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.61117.log.gz 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46189.log 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40467.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40467.log.gz 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30019.log 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46189.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46189.log.gz 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54583.log 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30019.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30019.log.gz 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65212.log 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.54583.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.54583.log.gz 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49629.log 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65212.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.65212.log.gz 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37879.log 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49629.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49629.log.gz 2024-09-22T03:38:39.900 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62338.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.37879.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37879.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49549.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.62338.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.62338.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66752.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.49549.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49549.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67453.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66752.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66752.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45446.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67453.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.67453.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58836.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45446.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45446.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53260.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.58836.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.58836.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44858.log 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53260.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53260.log.gz 2024-09-22T03:38:39.901 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55029.log 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44858.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44858.log.gz 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69175.log 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55029.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55029.log.gz 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53351.log 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.69175.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.69175.log.gz 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.53351.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.53351.log.gz 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65917.log 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38277.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.65917.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.65917.log.gz 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30509.log 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42693.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-09-22T03:38:39.902 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66446.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.30509.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30509.log.gz 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48320.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66446.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.66446.log.gz 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41699.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48320.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48320.log.gz 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34873.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41699.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41699.log.gz 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39386.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34873.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34873.log.gz 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47353.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39386.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39386.log.gz 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48905.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47353.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47353.log.gz 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64854.log 2024-09-22T03:38:39.903 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48905.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48905.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31328.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.64854.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.64854.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66175.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.31328.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31328.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51639.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.66175.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.66175.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33872.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.51639.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51639.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44098.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33872.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33872.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63848.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44098.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44098.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47213.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.63848.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.63848.log.gz 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-09-22T03:38:39.904 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47213.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47213.log.gz 2024-09-22T03:38:39.905 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67495.log 2024-09-22T03:38:39.905 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.48123.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-09-22T03:38:39.906 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57358.log 2024-09-22T03:38:39.906 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.67495.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.67495.log.gz 2024-09-22T03:38:39.906 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39888.log 2024-09-22T03:38:39.906 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.57358.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.57358.log.gz 2024-09-22T03:38:39.906 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55906.log 2024-09-22T03:38:39.906 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39888.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39888.log.gz 2024-09-22T03:38:39.906 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.55906.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.55906.log.gz 2024-09-22T03:38:39.947 INFO:teuthology.orchestra.run.smithi129.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.34331.log.gz 2024-09-22T03:38:40.017 INFO:teuthology.orchestra.run.smithi129.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-22T03:38:40.249 INFO:teuthology.orchestra.run.smithi129.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.53510.log.gz 2024-09-22T03:38:40.323 INFO:teuthology.orchestra.run.smithi129.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.29655.log.gz 2024-09-22T03:38:40.363 INFO:teuthology.orchestra.run.smithi129.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.58401.log.gz 2024-09-22T03:38:46.780 INFO:teuthology.orchestra.run.smithi144.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-22T03:38:46.856 INFO:teuthology.orchestra.run.smithi144.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-22T03:38:46.857 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-22T03:38:46.857 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m7.753s 2024-09-22T03:38:46.858 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m14.841s 2024-09-22T03:38:46.858 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.672s 2024-09-22T03:38:47.356 INFO:teuthology.orchestra.run.smithi129.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-22T03:38:48.390 INFO:teuthology.orchestra.run.smithi129.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-22T03:38:48.392 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-22T03:38:48.392 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m9.334s 2024-09-22T03:38:48.392 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m22.069s 2024-09-22T03:38:48.392 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m1.435s 2024-09-22T03:38:48.392 INFO:tasks.ceph:Archiving logs... 2024-09-22T03:38:48.393 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757/remote/smithi129/log 2024-09-22T03:38:48.394 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T03:38:51.584 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757/remote/smithi144/log 2024-09-22T03:38:51.585 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T03:38:53.482 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-22T03:38:53.491 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-22T03:38:53.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T03:38:53.494 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T03:38:53.583 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-22T03:38:53.593 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-22T03:38:53.593 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:38:53.597 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:38:53.611 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:38:53.614 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:38:53.616 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:38:53.616 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-22T03:38:53.616 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 -190us[ -188us] +/- 56ms 2024-09-22T03:38:53.616 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 +647us[ +649us] +/- 80ms 2024-09-22T03:38:53.616 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -301us[ -301us] +/- 65ms 2024-09-22T03:38:53.616 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:38:53.619 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:38:53.619 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-09-22T03:38:53.619 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 -192us[ -191us] +/- 56ms 2024-09-22T03:38:53.619 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 +641us[ +641us] +/- 80ms 2024-09-22T03:38:53.619 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -300us[ -300us] +/- 65ms 2024-09-22T03:38:53.619 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:38:53.620 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-22T03:38:53.628 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-22T03:38:53.628 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-22T03:38:53.651 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-22T03:38:53.742 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-22T03:38:53.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:38:53.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:38:53.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-22T03:38:53.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:38:53.867 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-22T03:38:53.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-22T03:38:53.901 INFO:teuthology.task.internal:Duration was 1384.945774 seconds 2024-09-22T03:38:53.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-22T03:38:53.935 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-22T03:38:53.935 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T03:38:53.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T03:38:53.975 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:38:53.977 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:38:54.190 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-22T03:38:54.190 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-22T03:38:54.191 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T03:38:54.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-09-22T03:38:54.245 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T03:38:54.273 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-22T03:38:54.274 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T03:38:54.286 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T03:38:54.340 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-22T03:38:54.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T03:38:54.352 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T03:38:54.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-22T03:38:54.494 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-22T03:38:54.494 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T03:38:54.518 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T03:38:54.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-22T03:38:54.553 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:38:54.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:38:54.582 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-22T03:38:54.607 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-09-22T03:38:54.634 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:38:54.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:38:54.660 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T03:38:54.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:38:54.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-22T03:38:54.701 INFO:teuthology.task.internal:Transferring archived files... 2024-09-22T03:38:54.702 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757/remote/smithi129 2024-09-22T03:38:54.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T03:38:54.736 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:56:02-crimson-rados-squid-distro-crimson-smithi/7914757/remote/smithi144 2024-09-22T03:38:54.736 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T03:38:54.768 INFO:teuthology.task.internal:Removing archive directory... 2024-09-22T03:38:54.768 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T03:38:54.772 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T03:38:54.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-22T03:38:54.831 INFO:teuthology.task.internal:Not uploading archives. 2024-09-22T03:38:54.832 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-22T03:38:54.852 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-22T03:38:54.852 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T03:38:54.854 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T03:38:54.867 INFO:teuthology.orchestra.run.smithi129.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 03:38 /home/ubuntu/cephtest 2024-09-22T03:38:54.873 INFO:teuthology.orchestra.run.smithi144.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 03:38 /home/ubuntu/cephtest 2024-09-22T03:38:54.874 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-22T03:38:54.884 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-22T03:38:54.955 INFO:teuthology.run:Summary data: description: crimson-rados/thrash_simple/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore thrashers/simple thrashosds-health workloads/radosbench-high-concurrency} duration: 1384.945773601532 flavor: crimson owner: scheduled_teuthology@teuthology success: true 2024-09-22T03:38:54.956 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T03:38:55.041 INFO:teuthology.run:pass